Putting Developers to the Test

Whiteboards and manhole covers won't find you a great programmer

Let’s imagine for a moment that you’re one of the world’s greatest chefs. You are a graduate of the CIA, have run a four-star restaurant, and had your own show on the Food Network. Now you’re interviewing to run the cafeteria at the hottest new startup in Silicon Valley (they’re offering major equity). After chatting for a bit with the CEO, she leads you outside the building. “I want to see how you work,” she says. “Cook me a meal.”

“Ok,” you respond, “where’s the kitchen?”

“Oh, no. I want you to find firewood in the park over there, build a fire ring, start a fire by rubbing some sticks together, then make a spear and hunt down a deer and cook it on the fire.”

I’ve recently had the pleasure of going through the job interview gauntlet with a few companies, and that’s essentially the process that most firms use to evaluate potential employees. They sit you down in a conference room, have someone pose you some kind of programmatic brain teaser, and then expect you to work it out on a white board.

This is so far removed from the realities of what a software engineer actually does today that you might as well be asking the candidate to sketch a portrait of the interviewer. I’m relatively fluent in at least dozen languages, but I don’t try to keep everything in my head at once. If I’m coding in Java, I’m using Eclipse. If I’m working on iOS apps, I’m in Xcode. I constantly hit command-space to autofill method signatures. I hover to get Javadoc. I search the web for code fragments.

Larry Wall says that one of the virtues of a programmer is laziness; do the least work that is needed to get the job done. When you’re hiring a developer, what you want to know is how efficient he is, and how good his code will be. If he can find the solution to a problem in five minutes of searching, it’s better than having to grind for hours trying to solve it without searching. Whiteboards don’t test that.

The single best job interview I ever had was with ITA (now part of the greater Google hegemony). They sat me down in front of a PC, posed a development problem, and let me go. The interviewer sat next to me for the two hours or so that I was coding. I was free to download the IDE of my choice, search the web, do basically anything I would have done on the job. At the end of the interview, what had they learned about me?

  • If I knew how to download and install the right tools to solve the problem
  • How I approached a development problem
  • How I looked for relevant information to help me solve problems
  • What my coding style was like

By comparison, a white board challenge only tests how well you can dredge up information in isolation from all the tools that a modern developer depends on. Unless the position you’re applying for is a job on a desert island with no Internet, it’s next to useless as a metric of how good a developer you really are.

Least you think I’m alone in my opinion, Google has pretty much thrown up their hands and admitted that their brainteasers were utterly useless in determining the quality of applicants. The best indicator of how well someone will work is to watch them work. Two hours of pair programming will tell you more about an applicant than an eternity of white board challenges.

A Commencement Speech for Graduating 2013 CS Majors
Talk Amongst Yourselves
Practice Makes Programmers

tags: , ,