entp hoth blog - Home

Fun Tender stats

2008-11-24T10:31:00Z

I’m writing some hacks to get a neat new feature for Tender out the door before we launch into a more public phase.

Here are the most common words used in tickets for Lighthouse:

["not", 258], ["we", 258], ["would", 260], ["if", 264], ["are", 276], ["can", 280], ["as", 291], 
["account", 301"], ["ticket", 304], ["my", 339], ["with", 347], ["but", 360], ["lighthouse", 380], 
["have", 432], ["be", 452], ["this", 498], ["that", 573], ["of", 599], ["in", 656], ["is", 665], 
["for", 710], ["it", 752], ["and", 803], ["a", 1277], ["i", 1295], ["to", 1748],

Awesome.

GitHub Uses Tender

2008-11-20T22:47:00Z

If you read Ruby blogs, you probably know that GitHub is run and built by some of the best programmers Ruby has. So we're pretty stoked to get an endorsement like this.

Custom fields? We don't need no stinking custom fields

2008-11-18T00:09:00Z

One of the most discussed feature requests for Lighthouse is the addition of custom fields. But given the goals of Lighthouse, it’s extremely unlikely that will ever happen. Why is that you say? Well – custom fields just don’t work, and here’s my argument for why that’s the case.

A little background

Before my work with ENTP, I worked at one of those big-bad-agencies for almost four years as a web developer. My job was not to write backend code. My job was not to make pretty designs. My job was to make it all work. Write the HTML, CSS, Javascript given designs from the design team and integrate it into Java/.NET/PHP backend systems. I was marrying the ideals from design and programming into the real world of a front-end user experience. Ideals don’t transform too well into the real world, so I ended up dealing with a lot of bugs.

A lot of bugs. During one of my projects, I estimate I was spending between 2-4 hours a day dealing completely with organizing bugs in our bug-tracking system, JIRA. JIRA had everything — access control, customizable workflows, priorities, rankings, modules, milestones and yes — custom fields. But at the end of the day, none of those features amounted to anything. It was just overhead that mandates more man-hours to organizing the bug tracker.

Organizing bugs is a myth

The basic idea behind custom fields is that you will be able to better organize your bugs and create complicated prioritized lists of work to be done. You’d be able to add a priority field, severity field, or a homeland security threat level color. The old prioritization system in JIRA I worked with prioritized bugs with two custom fields – priority & severity. We had three methods of making prioritization lists:

Priority - This was the priority of the bug, supposedly set by the client
Severity - This was the severity of the bug – whether it stopped all functionality, or was just a nitpick in the interface, for example
Project Manager coming over, slamming their laptop on my desk and telling me to fix issue X.

Guess which method worked best? If you guessed #3 – you’re a winner. For all the prioritizing and severitizing (which costs a lot of time during bug input) the best method of bug sorting was human communication (WHAT?!).

Bugs really only have three levels of prioritization

Just because custom fields don’t work doesn’t mean you need to give up on prioritization completely. Simple prioritization can be a huge win for developers in allotting their time. But you have to realize that there are really three levels of priority for a software bug.

FIX IT NOW — these are showstopper bugs. The server is crashed, the code you just deployed is eating up $1500 in sales per minute. These are solved by phone calls, in-person sit-downs, raving emails and caffeine. No bug tracker prioritization will help you. Enter the issue into a bug tracker for reference, but the priority is right now.
Needs to get done very soon. These are bugs that break some non-vital functionality or produce a bad user experience and need to get fixed as soon as possible. We deal with these issues by tagging them ‘@high’ in Lighthouse. You should have absolutely no more than a dozen of these bugs at any given point. Any more than that will degrade to noise and developers will ignore.
Other bugs. Any other bugs should be prioritized via milestones and assignees.

Any other prioritization should be considered a recommendation and therefore as little time as possible should be spent creating the prioritized list.

The human brain is more boolean than you might think

Project managers seem to have a myth that the human brain works like a computer – where it can align priorities in a list and work on the issues in that order. Adding more developers to the project should work like parallel processors – they all respect the priority list and work on issues in parallel in order of importance.

The problem is that the human brain does not work that way. It decides what it wants to work on, makes it’s own judgement of it’s value, and makes it’s own decision. Realistically – this falls into one of two categories: needs to be done, can wait. This is why Getting Things Done (GTD) systems have a concept of things that need to be done today, and things that need to be done next.

If there is nothing that needs to be done, then people may reference a priority list as a recommendation – but they will ultimately decide for themselves. They may chose to clear out 50 easy browser bugs (low in priority) versus the one hard bug (high in priority) if nothing is urgent. Think about bugs like chores – do you take the trash out when it’s almost full, or the day before trash day?

Complicated prioritization lists are just false security

For all my arguments against custom fields, there will be plenty of people arguing the opposite. As a client-facing project manager, it’s very easy to think that you need to have complicated prioritization. You can produce lists of prioritized bugs for clients to look at and gush over. They are a wonderful source of false security — everyone not doing the work thinks everything is on track. “Look, we only have 5 high priority, showstopper tickets impacting the services layer for next month’s release left!” But the people on the ground working on the code know that’s not the case. Developers see the project as a whole.

Just because you can create a complicated hierarchical, phased priority list does not mean your developers will work on them in that fashion. I think most people underestimate the impact of personal communication and overestimate the value of technology. Bug trackers are a good thing, but only if they function with your team.

At Lighthouse we’re determined to make the best possible product – and we think the absence of a feature can be a feature in itself. Try working without custom fields for a little while and you’ll soon see how the glove fits.

Next up: How we work with Lighthouse - an eye into how ENTP uses it’s own bug tracker.

Sinatra: ActiveRecord lib/models File

2008-11-13T20:15:00Z

I've got a Sinatra app using the same Active Record DB as a much larger Rails app. I need code to handle that, and I also want to keep my Sinatra app clean.

It's kind of a cheat, and for some reason lib/controllers doesn't work, but it's a tidy way to sneak a little naughty complexity into Sinatra's one-file simplicty.

Improving my git workflow

2008-11-10T21:44:00Z

I'm a fan of "rip and tear" refactoring. Dive in, hack and slash, see what works. Whether my hack-and-slash-fest lasts a few minutes or a few hours, I always do my dirty work in a git 'topic branch'.

A topic branch is simply:

git checkout -b trevor-287-date-formatting
# hack and slash

NB: at ENTP we name our branches according to committer, Lighthouse ticket #, and a short descriptive string.

At every step along the way, when I think "okay, that works", I commit. That way, as I continue on my path of destruction, any dead ends I encounter can be undone with a simple "git checkout -- ." - a project-sized undo buffer, if you will.

Sometimes I'll even branch off of my topic branch to try out ideas. Cheap to create, cheap to destroy.

So topic branches and frequent commits are my safety net. They let me be as aggressive as I want in my changes and help me to maintain 'flow'. Except...

Except for when it comes time to publish my topic branch for peer review. I may be a hack-and-slash bandit but at ENTP we've got some pretty stringent requirements for making sure code "makes the grade" before it even hits the staging servers, let alone production.

That means that every change gets pushed to a remote branch and reviewed by at least one other brainiac as a first step. And this is where my lovely topic branch has let me down, just a little.

Pushing a branch to remote is easy:

git push origin trevor-287-date-formatting

The only problem is .git/config isn't updated so that I can do a simple "git pull" on those rare occasions Giles or Matt would have the audacity to correct my finely-crafted code. Until now I've been hand-editing .git/config to make sure everything works. Blech.

What I've wanted is a way, with a flick of the keyboard, to 'promote' my topic branch of trevor-287-date-formatting to a tracked remote branch of the same name.

And while I could easily start the whole process with:

# create the new remote branch based on current master:
git push origin master:refs/heads/trevor-287-date-formatting
# create a local tracking branch of trevor-287-date-formatting
git branch --track trevor-287-date-formatting origin/trevor-287-date-formatting
# start working on it:
git checkout trevor-287-date-formatting

It just doesn't feel fluid enough to me. Remember, topic branches appear and disappear from my local repo faster than stock value on the Dow Jones.

Instead, I wrote a shell script that encapsulates my manual steps (push to remote, amend the config) into a concise command. Much more satisfying.

I present to you: git-promote

#!/bin/sh
#
# Promotes a local topic branch to a remote tracking branch of the same name,
# by pushing and then setting up the git config

curr_branch=$(git symbolic-ref -q HEAD | sed -e 's|^refs/heads/||')

remote_branch=$(git branch -r | grep "origin/${curr_branch}")
[ -z "${remote_branch}" ] && ( git push origin "${curr_branch}" )

origin=$(git config --get "branch.${curr_branch}.remote")
[ -z "${origin}" ] && ( git config --add "branch.${curr_branch}.remote" origin )

merge=$(git config --get "branch.${curr_branch}.merge")
[ -z "${merge}" ] && ( git config --add "branch.${curr_branch}.merge" "refs/heads/${curr_branch}" )

Assuming you have $HOME/bin in your PATH, put that script in a file called $HOME/bin/git-promote and type this on the command line:

chmod 755 $HOME/bin/git-promote

Now you can promote your topic branches to remote tracking branches like this:

# just to ensure sure my working branch is the one I want to promote...
git checkout trevor-287-date-formatting
git promote

Done and Done.

If anyone has better suggestions, do let me know. I'm always keen to learn about other people's workflows.

Ack In Project (TextMate plugin)

2008-11-09T19:51:00Z

ENTP programmers produce a lot of open source code. One cool project you might have missed: Trevor Squires' TextMate plugin which allows you to use ack. Ack In Project is like Find In Project, only faster, and acker.

For more detail and a higher-quality version of the screencast, check out Trevor's blog.

Comments, Code Smells, And Clients

2008-11-07T21:41:00Z

Neal Ford writes that comments are a code smell:

We were on a project where the client insisted on Javadoc comments for every public class and method. We started the project adding those comments, but eventually stopped. When doing agile development, you don't want anything that hampers refactoring. Having comments in place caused a dilemma: do I refactor the method and change the comment (the most work), refactor the method and leave the old comment (with the theory being that I'll change it again later, and would rather just have to update the comment once), or not refactor? No good options here. Having pervasive comments discourages refactoring because it adds significant extra friction. On this project, we abandoned commenting as we went along. The last week of the project we literally did nothing but go back and add comments to the code, which worked well because the code base had settled down by that point. But notice what's lurking in wait: the client wanted all the comments there to make it easier to maintain in the future. But who's to say that whoever maintains that code will keep the comments up to date?

Neal's post is very worth reading. He says the Lysol to clear away this particular smell is made of two things: automated tests and composed method, which is Kent Beck's name for the technique of creating large numbers of very small methods with very clear names. He also offers an explanation to one interesting mystery, that of why this approach is so much more popular in Ruby and Smalltalk than it is in Java: dynamic languages encourage descriptive method names. Check out the comments on Neal's post; he didn't explain that in much detail, so I asked him for follow-up. My guess is that he sees this as a side effect of the more natural syntax dynamic languages use.

The quote I pulled out raises a different issue: what do you do when your client wants something you can't recommend? Like any consultancy, ENTP encounters this question every once in a while. Neal's example is a great example because it's not cut and dried. The Javadoc comments may have established an obstacle to refactoring and clean maintenance; on the other hand, they resulted in extensive documentation, and extensive documentation is a good thing.

My personal answer is that you let the client know, but you also let the client decide. I'm not sure what the official ENTP answer is. The official ENTP answer could be different on comments, too. I feel the same way about comments in code as I feel about adverbs in writing. In a perfect world, it would never happen, but in reality you have to give way occasionally.

The new entp.com

2008-11-07T02:48:00Z

Today we launched the new entp.com – and I’d like to walk through some of the design thoughts and methodology we used. First things first: entp.com was designed and built inside the span of ten days (in the midst of client work, too!). We’re a firm believer in doing instead of talking about doing – once we had it in our minds to get started, we worked to make it happen as fast as possible.

The Challenges

Redesigning any agency site is always one of the hardest tasks a designer can take on. Not only do you have the world’s worst client (yourself) – but you’ve got the worst critics (your fellow designers) too. The biggest challenge with ENTP was just exactly how to portray the company, the work, and the people in it.

Typical agencies have life pretty easy – they hire almost exclusively full-time employees, force them to sign over their lives, and then take credit for everything that an employee thinks during their employment. ENTP is a bit different – a large portion of people who work here are contractors, and ENTP allows (promotes!) people to work on their own side-projects. It makes for a very complicated relationship when you try and explain the relationship between something like Archaeopteryx and ENTP. ENTP doesn’t own any rights, but Giles does work here – and I think that’s a credit to the company’s ability. The same can be said for the dozens of plugins and libraries created by ENTP employees. You would struggle to find a single Rails developer who hasn’t used code from an ENTP employee before.

Another hurdle in the way of ENTP is that as individuals each of us are fairly well known in our communities – but as a company few people have ever heard of us. The struggle in designing the site was to reign in all of these disparate ideas and combine them into one solidified ENTP image.

The Approach

At ENTP, we approach design a little differently than everyone else. The design we ended up with was a true collaboration between Justin & myself. We spent a couple of days tossing PSDs back and forth between each other and then threw up a few options to the ENTP Campfire room to make the final decisions. The result is a design that keep a great focus between what we do, what we make, and who we are.

The result is this evolution of the home page:

Simply Static

Hanging out with a group of developers, it’s hard to convince people that sometimes a static lifestyle is the way to go. For simple brochure sites – I still think 100% static is the best method out there. But there are some challenges associated with this – for example, I wanted to pull in people’s github repositories and rotate out employees at the bottom. Luckily for me, Github offers a nice JSON API with callbacks – so I’m able to pull in everyone’s non-forked repositories (thanks to a quick bribe) and show them there. Storing everyone’s information in a nice static json file allows me to work the rest of my magic, all with Javascript. Our deployment strategy is a cron job running a git pull – so anyone with access to the git repository can edit content and push it live.

For simple sites, static is the way to go – don’t be afraid to under-engineer your solution if you need to.

Meet ENTP

Now that we’ve finally got (almost) all of our team up, you can get an idea of who entp really is – you might be surprised to see who’s been working here.

What "git add" Really Means

2008-11-05T21:56:00Z

Not being a git expert, I still sometimes stumble on straightforward things. Recently I did a git checkout and ran into merge difficulties. One version of a spec had a particular example commented out; another version didn't. I wrestled git for a while, trying to find a way to tell it, "Dude I don't care, pick one file or the other so I can get on with my day." Neither version of the file mattered, because after a day or two more of work, the branch won't need the file at all.

After searching for a simpler solution, I cleaned the file up by hand, did git add filename, plus git commit filename, and I was done. Some of the messages I got during the process were confusing, however, and I didn't remember right away that you're supposed to git add a file that git already knows about.

The first time you use git, people tell you, first you git add the new files git needs to know about, and then you git commit those files. It's pretty easy to leap to the conclusion that git add means "add this file to the list of things git cares about," which makes this use case a little counter-intuitive, or at least less obvious than I would have liked. It's kind of like git has Alzheimer's and you have to introduce it to the same new person twice.

A git tutorial on the GNU Smalltalk blog has part of the answer:

What git add does is to move the current version of the named file to a special staging area, holding files that are ready to be committed

The GitHub blog reports that git add may soon take on the new name git stage, to reflect this.

I’m excited that staging files may soon be done via ‘git stage’ rather-than/in-addition-to ‘git add’. This is nice for new users who often have a hard time seeing why you have to keep ‘git add’ing to stage your changes.

The GNU Smalltalk git tutorial's interesting and worth reading. I also stumbled across an awesome feature I hadn't known about: git add (aka stage) has an interactive console like IRB or the Rails console. You can also use it for commits. I'm going to give that a whirl and I'll post here if I get any interesting results.

Craziness Considered Irrelevant

2008-10-31T19:32:00Z

ENTP has some of the best programmers in the Rails community. I'm lucky to be a part of a group like this. Some companies organize around best practices and competent coders. That's great, but it's not enough - it's middle-of-the-road. ENTP's more like the Avengers or the Justice League: a team of unusual people who rock at their individual specialities.

Recently I posted something controversial on my personal blog, and somebody criticized ENTP on Twitter in response. That's unjustified. My views go on my blog, and ENTP's views go on ENTP's blog. I think anybody who uses the Internet understands this. If you need it clarified, consider that the same page on my blog where my rant appeared also featured a blog post entitled Cannibal Baby: The Love Story. That is not an ENTP-sponsored post either.

Courtenay, ENTP's founder, gives this as his official position:

I formed ENTP after reading a book, Lucky or Smart? by Bo Peabody. It's a tiny book, and really you only need to read the first chapter. Following his inspiration, I went out to find the smartest motherf*ckers in my field, get them together, and let them do their thing.

We give people a free reign to do pretty much whatever they want: that's the point. Recently Giles wrote some batshit crazy post, on his own blog. Some people thought that it reflected poorly on our company. I had a much longer post written, but the summary of it is this: we know that Giles is crazy, we knew that when we hired him. Giles is a foundry of many, many awesome ideas. He also has some terrible ones. The opinions and words he writes on his own blog are his alone, and while I personally disagree with some of them, his good ideas are really good, and he's a valuable asset to our company.

Courtenay isn't the only person at ENTP to disagree with my controversial stance. Only one person at ENTP said that they thought any part of my post seemed even slightly reasonable at all. Even if they had agreed, it would have had nothing to do with the company. Like Courtenay says, I have a reputation for taking controversial stances and creating innovative projects. They hired me for the innovation, and they knew they'd have to take the crazy along with it.

What's Hot On GitHub: ENTP

2008-10-31T08:49:00Z

Ruby Inside's monthly What's Hot On GitHub? post features two projects this month by developers at ENTP: astrotrain and seinfeld.

Astrotrain

Astrotrain is an experimental e-mail to Web router. It receives incoming mail and turns it into HTTP POST requests. If you've seen the innovative and intuitive way Tripit folds e-mail into its user interface, Astrotrain's a similar kind of project.

Seinfeld

Seinfeld is the code which powers Calendar About Nothing, which is based on a classic LifeHacker post about Jerry Seinfeld.

[Seinfeld] said the way to be a better comic was to create better jokes and the way to create better jokes was to write every day. But his advice was better than that. He had a gem of a leverage technique he used on himself and you can use it to motivate yourself—even when you don't feel like it.

He revealed a unique calendar system he uses to pressure himself to write. Here's how it works.

He told me to get a big wall calendar that has a whole year on one page and hang it on a prominent wall. The next step was to get a big red magic marker.

He said for each day that I do my task of writing, I get to put a big red X over that day. "After a few days you'll have a chain. Just keep at it and the chain will grow longer every day. You'll like seeing that chain, especially when you get a few weeks under your belt. Your only job next is to not break the chain."

"Don't break the chain," he said again for emphasis.

Lighthouse/GitHub Integration Screencast

2008-10-30T19:55:00Z

Cameron Westland from BigBang Technology posted a screencast demonstrating how to use Lighthouse and GitHub web hooks to view your GitHub changesets from within your Lighthouse project.

Integrating Github with Lighthouse has been a great time saver. It has allowed us to stay on top of fixes and keep track of everything we're doing in one place. We don't need to have everyone looking through the git logs now! This short screencast shows you how to get things set up.

These features are easy to use and make your Lighthouse experience better. Check out the screencast - it's short, clear, and to the point.

Add Lighthouse tickets to your GitHub wiki

2008-10-07T00:15:00Z

In case you've missed it, we're drinking the Github Kool-Aid extensively. One request we've heard quite often is a way to show your Lighthouse tickets on your Github wiki. Check out the newly minted Lighthouse badge. Just include a little bit of code and the JavaScript file into your wiki and you have your list:

<span style="display:none" id="lighthouse-parameters"  account="activereload" query="state:open">YOUR API KEY</span>
<div><script src="http://github.com/Caged/lighthouse-badge/tree/master/src/lighthouse-badge.js?raw=true"</script></div>

Because Github doesn't allow inline script blocks, you specify your parameters with a hidden span.

Checkout the README for full instructions and be sure to watch the project to keep up with changes. This is a preliminary version that uses jQuery, but we plan on expanding this in the days to come.

Warehouse is now open source

2008-09-30T17:56:00Z

After a long vacation, we’ve decided to release Warehouse as open source. The fact is, we (and most of our target audience) moved from subversion to git or mercurial. Also, the Logical Awesome guys scare us. First, there’s their kick ass git commit browser out there that embodies the spirit of git exceptionally well. Then, they prove that they can do Subversion hosting better than us. Yikes.

What happens to the current customers? We’ll be issuing refunds out for anyone that’s purchased in the last two months (that’s as far as Paypal will go back). Anyone else that feels they should get a refund can contact me and we’ll work something out. We’ll still be providing support at the forums. It will be getting its own Voxpop instance soon as well. Oh yes, don’t forget to remove the license check on your existing warehouse installs. We don’t want to leave you stranded like Walmart music customers.

There’s still a need for something like Warehouse though. Something localized for your internal git servers. We actually need one for the internal ENTP gitosis server, so there’s work started on a git/svn hybrid version. I’ve spoken with several people interested in this, so I’m hoping we can get this finished up relatively soon. It’d also be really cool if someone could add mercurial support :)

We’re going with the AGPL license and have the source (both the unreleased master branch and the latest 1.1.6 release) on github. Thanks for all the support from the Warehouse customers, and happy forking!

Vox Populi

2008-09-22T23:24:00Z

ENTP's launching something new soon. It solves a problem every ticket-tracking system has.

Here's the scenario: User A finds a bug, submits a ticket. User B finds the exact same bug, submits another ticket. User C finds completely different behavior in the UI which traces back to the exact same bug, submits another ticket. Tech Support Person goes, oh, these are all the same ticket, and closes out two of the three.

User B and User C complain about their tickets being closed. They're still seeing the bugs, so they open new tickets. Which Tech Support Person then closes again. The cycle repeats until the bug's eliminated, or until the heat death of the universe.

FAIL.

At ENTP, we're fans of Post-It Notes.

The brilliant thing about the Post-It Note: simplicity.

The simple solution to this age-old ticket-tracking problem is to say there's a difference between bug reports and tickets. With Vox Populi you're going to have a user-oriented front end to Lighthouse that separates user observations from task scheduling. You won't have to care if other tickets exist already. If you see something, say something.