- On the NSA — intelligent unpacking of what the NSA crypto-weakening allegations mean.
- Overview of the 2013 OWASP Top 10 — rundown of web evil to avoid. (via Ecryption)
- Easy 6502 — teaches 6502 assembler, with an emulator built into the book. This is what programming non-fiction books will look like in the future.
- Kochiku — distributing automated test suites for faster validation in continuous integration.
ENTRIES TAGGED "devops"
Velocity 2013 Speaker Series
I want to start by thanking John and Steve for the warm welcome. They’ve created something very amazing with Velocity, and I’m excited to be a part of it.
It might seem a bit odd to talk about What’s Next at the beginning of a conference, but I figure the best time to go to the bank and ask for a loan is when you actually have some money.
What we’ve been talking about at Velocity, especially the DevOps side of things, is only the tip of the iceberg when it comes to how businesses are changing. And that shift is from the sequential to the concurrent. It used to be that we threw things over a series of walls, from Product Management to Design, to Development, to QA, to Production, to Customer Service and so on. That was an old world of software and one-year development cycles.
How you handle failure can mean the difference between "just another incident" and a revenue-stealing accident.
I was ready to get home. I’d been dozing throughout the flight from JFK to SFO, listening to the background chatter of Channel 9 as a lullaby. Somewhere over Sacramento, the rhythmic flow of controller-issued clearances and pilot confirmations was broken up by a call from our plane:
“NorCal Approach, United three-eighty-nine.”
“United three-eighty-nine, NorCal, go.”
“NorCal, United three-eighty-nine, we’d like to go ahead and…”
My headphones went silent, Channel 9 shut off.
I didn’t think too much of it as we continued our descent, flight attendants walking calmly through the cabin, getting us ready for landing. I had noticed our arrival path was one I was unfamiliar with, but nothing else seemed out of the ordinary… until we turned onto the final approach. In the turn, I noticed the unmistakable glint of firetrucks’ rotating red lights, lined up alongside the runway.
Explicit expectations are key to operating at scale
Our lives are rife with expectations.
When we flip the light switch, we expect electrons to flow; when we issue CPU instructions, we expect to get the correct answer; when we look at commit logs in the source repository, we (hopefully) expect tests to accompany them and that our colleagues have run them, pre-checkin. But we’ve all probably been burned by these types of assumptions at some point.
In an operational environment, like the large scale websites and build farms we’re responsible for, these sort of expectations can be a costly cause of errors, and are one of the prime sources of miscommunication. Many a postmortem has uncovered that some expectation the ops team had of the development team was actually an assumption… and we all know that old saying about assumptions and donkeys.
In the operational environment, miscommunication can be costly; but there are some easy ways to improve it.
Editor’s note: This is part two in a four-part series on the “-ations” of aviation that can provide further insight into DevOps best practices and achieving them. Part one, on how standardization helps organizations scale and is actually a part of healthy DevOps culture, can be read here.
Communication is an enigmatic topic when it comes to engineering. Parts of our jobs—blueprints, chemical formulae, and source code—require extremely precise forms of communication (even if it doesn’t end up communicating to the steel, molecules, or silicon what we intended). But when it comes to email threads sifting through requirements, meetings about implementation styles and risk assessment, and software design documentation, we often fumble.
Let’s face it: there’s a reason the “engineer equals bad communicator” stereotype exists. But there are some simple things that can be done, both individually and technologically, to begin challenging that stereotype.
Dual Navigation Receivers Required
There are obviously many forms of communication. In an operational context, it’s useful to distinguish between static and active communication.
Standardization done right can save your sanity and improve your culture
Capital-P “Process” ™ is something many software developers, operations engineers, system administrators, and even managers love to hate.
It is often considered a productivity-killing, innovation-stifling beast whose only useful domain is within the walls of some huge, hulking enterprise or sitting in a wiki nobody ever reads.
I have always found distaste for process fascinating and now even moreso that configuration management and version control have become such core tenets of the DevOps movement. The main purpose of those tools is to provide structure for software development and operations to increase reproducibility, reliability, and standardization of those activities.
NSA Crypto, Web Traps, Learn by Doing, and Distributed Testing
However one defines "enterprise," what really matters is an organization's culture.
Bill Higgins of IBM and I have been working on an article about DevOps in the enterprise. DevOps is mostly closely associated with Internet giants and web startups, but increasingly we are observing companies we lump under the banner of “enterprises” trying — and often struggling — to adopt the sorts of DevOps culture and practices we see at places like Etsy. As we tried to catalog the success and failure patterns of DevOps adoption in the enterprise, we ran into an interesting problem: we couldn’t precisely define what makes a company an enterprise. Without a well understood context, it was hard to diagnose inhibitors or to prescribe any particular advice.
So, we decided to pause our article and turn our minds to the question “What is an enterprise, anyway?” We first tried to define an enterprise based on its attributes, but as you’ll see, these are problematic:
- More then N employees
- Definitions like this don’t interest us. What changes magically when you cross the line between 999 and 1,000 employees? Or 9,999 and 10,000? Wherever you put the line, it’s arbitrary. I’ll grant that 30-person companies work differently from 10,000 person companies, and that 100-person companies have often adopted the overhead and bureaucracy of 10,000 person companies (not a pretty sight). But drawing an arbitrary line in the sand isn’t helpful.
- Juju — Canonical’s cloud orchestration software, intended to be a peer of chef and puppet. (via svrn)
- Cultural Heritage Symbols — workshopped icons to indicate interactives, big data, makerspaces, etc. (via Courtney Johnston)
- Quinn Norton: Students as Hackers (EdTalks) — if you really want to understand the future, don’t look at how people are looking at technology, look at how they are misusing technology.
Retro Hackery, Etsy Ops, Distributed Identity, and lolcoders
- How Things Work: Summer Games Edition — admire the real craftsmanship in those early games. This has a great description of using raster interrupts to extend the number of sprites, and how and why double-buffering was expensive in terms of memory.
- IAMA: Etsy Ops Team (Reddit) — the Etsy ops team does an IAMA on Reddit. Everything from uptime to this sage advice about fluid data: A nice 18 year old Glenfiddich scales extremely well, especially if used in an active active configuration with a glass in each hand. The part of Scotland where Glenfiddich is located also benefits from near-permanent exposure to the Cloud (several clouds in fact). (via Nelson Minar)
- Who Learns What When You Log Into Facebook (Tim Bray) — nice breakdown of who learns what and how, part of Tim’s work raising the qualify of conversation about online federated identity.
- lolcommits — takes a photo of the programmer on each git commit. (via Nelson Minar)
Simplifying IT automation
IT infrastructure should be simpler to automate. A new method of describing IT configurations and policy as data formats can help us get there. To understand this conclusion, it helps to understand how the existing tool chains of automation software came to be.
In the beginnings of IT infrastructure, administrators seeking to avoid redundant typing wrote scripts to help them manage their growing computer hordes. The development of these inhouse automation systems were not without cost; each organization built its own redundant tools. As scripting gurus left an organization, these scripts were often very difficult to maintain by new employees.
As we all know by the huge number of books written on the topic, software development sometimes has a large amount of time investment required to do it right. Systems management software is especially complex, due to all the possible variables and corner cases to be managed. These inhouse scripting systems often grew to be fragile.