When you’re dealing with complex systems, failure is going to happen; it’s a given. What we do after that failure, however, strongly influences whether or not that failure will happen again. The traditional response to failure is to seek out the person responsible and punish them accordingly — should they be fired? Retrained? Moved to a different position where they can’t cause such havoc again?
John Allspaw, SVP of technical operations at Etsy and co-chair of the O’Reilly Velocity Conference, argues that this “human error” approach is the equivalent of cutting off your nose to spite your face. He explains in a blog post that at Etsy, their approach it to “view mistakes, errors, slips, lapses, etc., with a perspective of learning.” To that end, Etsy practices “blameless postmortems” that focus more on the narrative of how something happened rather than who was behind it, and that remove punishment as an outcome of an investigation.
“A funny thing happens when engineers make mistakes and feel safe when giving details about it,” Allspaw writes, “they are not only willing to be held accountable, they are also enthusiastic in helping the rest of the company avoid the same error in the future.” This is where the learning comes in: if you simply fire the person responsible, you’ve just removed the only expert source of information on how that particular failure occurred. And under the threat of punishment, Allspaw writes, the responsible party is “disincentivized to give the details necessary to get an understanding of the mechanism, pathology, and operation of the failure.” Which, he notes, pretty much guarantees that failure will occur again.
In this Radar Podcast episode, I sit down with Allspaw for an in-depth discussion on blameless postmortems and the importance of cultivating a just culture in an organization. We also talk about his current research into the gap between how we think works gets done and how it actually gets done. You can join the discussion in the comments or with Allspaw directly through his Twitter handle, @allspaw.