On Monday Neil McAllister posed the question “is the hacker ethic harming American developers?” Slashdot picked it up and Tim forwarded it to the Radar list. As you might expect, it resulted in some spirited discussion.
James Turner kicked things off with this response (it has been slightly edited from its email form). After James lays out his argument I’ll reply with my thoughts. Then we hope to hear from you. Let us know what you think.
I’ve worked in a lot of organizations that thought that the kind of rigid deforesting paradigms that Nayar is referring to were the magic bullet to keeping all three of the variable (dollars, time, features) under control. Without exception, all they did was get in the way and reduce the productivity of the most senior people to the level of the most junior. All of them exhibited some degree of failure, some catastrophic.
The India shops *love* methodologies like UML and the like, specifically because the problems have been reduced to a simplistic enough granularity that they can be doled out to junior-level staff, who may have only been onboard a few weeks because of the massive churn over there.
At least 3 times at 3 different companies, I’ve seen major pieces of work brought back in-house because the Bangalore team had fallen so far behind or proved so unable to get beyond the literal description of work that they were endangering the project. When you combine the time difference with a tendency to halt dead in their tracks as soon as they hit a stumbling block, it can be a recipe for disaster.
There are certainly some good Indian shops, and I know some outstanding Indian developers (most of whom have come to the States.) But I find Nayar’s comments hilarious. It’s akin to someone saying that American football players aren’t employable in Jamaica because they aren’t able to limbo well. Look at the most successful Web 2.0 companies today, most of them started as garage enterprises with a few talented developers, not a 60 person team of UML jockies following some Arthur Anderson project management program. Heck, look at Google Labs.
In huge projects, you obviously need some master planning and coordination to make sure the tracks meet at the right place to drive in the spike, but I don’t see any effort being made these days to right-size the amount of project overhead to the needs of the projects. Instead we get a one-size-fits-all approach that smothers anything but the largest project in paperwork. Even some of the original authors of the Agile manifesto, when I’ve talked to them, point out that part of being Agile is picking and choosing the right components of project management that make sense for a given task.
Nayar’s remarks are incredibly self-serving. “We’re the best, because we can mindlessly follow some arbitrary and flawed development process.” Or is he claiming that Indian projects do better QA? Not in my experience…
This entire debacle is representative of a problem I think is endemic in the industry these days, the inability or unwillingness to engage in rapid prototyping. Every successful project I’ve ever worked on (and I’ve worked on some fairly large enterprise-sized projects), we started by designing and coding a quick “throw-away” skeleton of the application, that let us look at how the thing worked, where the unseen warts were, and where the vendors had lied about their APIs, etc. This is the crucial and neglected stage in project design, one that most modern design paradigms ignore or actively discourage. Even Agile tries to jump in and start coding the finished application from the get go, although if project teams were willing to aggressively refactor (a tenant of Agile), early project work could be a rapid prototype (although the story model of scrum really doesn’t fit well with this, unless you make the prototype a story…)
This is also something I’ve never seen an offshored team do particularly well…
Well, I’ll be damned if I’m going to jump in and take the side that says hacking is bad for American programmers. First off, I don’t need that kind of flame bait and second, I don’t believe it. I think approachable programming is hugely important because that’s how many people get into the field in the first place. However, my reaction to the article was very different than James’ and I might as well try to explain.
I’m not going to take an opposing position, but it’s not really an orthogonal position either. Maybe it has a power factor of about .7 or so. Here’s my (also edited) response…
When I when I read McAllister’s piece, at least some of it resonated with me. Before we were bought by a large firm, we were a small company that grew from nothing to 250 people, about 200 or more of them were programmers. So, a whole lot of my time over three years was spent hiring programmers and building cohesive teams that could deliver to our customers.
In our hiring we aggressively hired hackers into the mix. We wanted outside-our-industry thinking and we thought they brought in creativity. We called it “hiring weird, but not weird weird.” Occasionally we pushed it to weird and a half. For our efforts we got creative problem solving and interesting (but frequently weird) dinnertime conversation when we travelled.
However, our pollyanna idea of “disciplined teams catalyzed with a bit of weird” didn’t always work out.
That leads to the bit that resonated with me: the sense that hacker = distilled essence of American individualism combined with lots of ISTJ Myer’s Brigg’s Type Indicator. Individualism is a trait that I hold dearly, but it can make a cohesive team effort difficult if people are unwilling to suborn themselves to the goals of the team. Remember those tee shirts the football team always wore at your university? “There is no I in team?” I sometimes joked that I was going to make a batch that said “I’m the Me in team.”
Maybe we were just growing fast and it was going to take more storming and norming than I had patience for, but at times it was a struggle to get everyone to see past their individual biases and focus on what we were trying to achieve, and we couldn’t do what we were trying to do with teams of one.
But it really wasn’t a hacker problem if hacker means self taught like McAllister implies. We had a lot of people with CS degrees and we used to talk a lot about whether and how their degrees had prepared them for their jobs.
Separate from the individualist approach to development, few of our recent graduates came to us prepared with the terminology and practices of any development approach (or engineering approaches like continuous integration etc.). They knew how to code, but not how teams coded.
At one point I gave a talk on agile software development to about 100 CS students at a university in Philadelphia and I asked them to raise their hands if they had ever done a team project with greater than two people on the team. I don’t recall anyone raising a hand. Then I asked if they had ever covered development methodologies in their classes and a few acknowledged they had, but it had been abstract classroom stuff only. That part surprised me.
I’m not sure that the “sanctity of engineering” argument really makes that much difference. I have little faith in McAllister’s scheme to do computer engineering instead of computer science coursework.
My undergraduate degree was in Mechanical Engineering and I can only imagine how useless I would have been to a firm that actually did engineering, and for mostly the same reasons. I knew how to take integrals and I still know the packing ratio of a hexagonal close packed material but I didn’t know squat about how a complex machine actually got designed in a team setting. It’s interesting to note that the Engineer in Training exam I took (a precursor to the professional engineer’s exam) didn’t probe my knowledge of team practices at all.
Maybe there just isn’t time in an undergraduate degree to teach everything that an engineer needs to know. Plus, can you imagine the drop out rate in CS/CE if ITIL was a required course?
Since James mentioned Google I’ll switch gears and muse about ecosystems for a moment… I guess I tend to bristle when I hear that everyone should just develop software the way Google does.
Google is to computing what LA was to Aerospace and Electronics in the early 60’s. It’s gravitational force attracts five sigma talent (probably a bunch of six too) in ways that the rest of us can only envy. More generally, Silicon Valley has had programmer talent flowing into it for the last twenty years the way Hollywood sucks pretty people out of the midwest.
Maybe it’s not as obvious because you can’t spot brains the way you can spot an oddly beautiful wait staff, but the valley has been the vortex of a talent-laden embarrassment of riches for a long time and, if you work there, you might not even notice it. However, I think that at some level this effects what kinds of processes work when you build software. I think it’s at least a little part of the reason why an ERP system in a manufacturing town gets implemented differently than MapReduce (there are other reasons too having to do with software as product vs software as supporting infrastructure). Combine that with the very clear shared vision of “lets do something great and get rich together” thing that valley firms often have, and well, it’s easy to see how smart people coalesce to build amazing stuff.
It’s easy to denigrate Arthur Anderson’s progeny or the offshore firms they compete with, but they do different work, with a different talent pool, for different ends, and with a very different set of personal and organizational incentives. Or, put another way, Kelly Johnson didn’t build the SR-71 with General Motor’s engineers, and General Motors didn’t design the Chevy Cavalier with the Skunkworks’ processes. However, even at the Skunkworks, Johnson’s brilliant engineers did conform to a process and work together as a team toward a shared vision. And, conversely, I bet a lot of talent is left on the table at General Motors because of processes too restrictive in their attempt to remove all uncertainty.
So,… maybe it’s possible that Google’s (or the valley’s in general) processes are appropriate to an ecosystem that, because of the intellectual environment and potential for riches, is rich in IQ and initiative. So it ends up feeling more “special forces” and less like “infantry regiment.” And over there closer to the hump in the normal distribution curve, or in a different cultural environment outside of the valley, a different flavor of processes may be effective?
The counter argument to that, which I’ll go ahead and provide, is that I once helped teach a team of engineers at midwestern defense contractor how to do agile development. The effect was amazing and immediate and their productivity and satisfaction went up tremendously; until their management freaked out and shut it down when they “perceived” that it created too much uncertainty in their processes.
Well, it’s obvious that I don’t know the answers here, so, with that, I’ll stop thinking out loud.
What do you think?