- On Being a Senior Engineer (Etsy) — Mature engineers know that no matter how complete, elegant, or superior their designs are, it won’t matter if no one wants to work alongside them because they are assholes.
- Control Theory (Coursera) — Learn about how to make mobile robots move in effective, safe, predictable, and collaborative ways using modern control theory. (via DIY Drones)
- US Moves Towards Open Access (WaPo) — Congress passed a budget that will make about half of taxpayer-funded research available to the public.
- NHS Patient Data Available for Companies to Buy (The Guardian) — Once live, organisations such as university research departments – but also insurers and drug companies – will be able to apply to the new Health and Social Care Information Centre (HSCIC) to gain access to the database, called care.data. If an application is approved then firms will have to pay to extract this information, which will be scrubbed of some personal identifiers but not enough to make the information completely anonymous – a process known as “pseudonymisation”. Recipe for disaster as it has been repeatedly shown that it’s easy to identify individuals, given enough scrubbed data. Can’t see why the NHS just doesn’t make it an app in Facebook. “Nat’s Prostate status: it’s complicated.”
ENTRIES TAGGED "devops"
Mature Engineering, Control Theory, Open Access USA, and UK Health Data Too-Open?
iOS Pentesting, Twitter's Infrastructure, JS Data Sync, and Chromium as C Runtime
- idb (Github) — a tool to simplify some common tasks for iOS pentesting and research: screenshots, logs, plists/databases/caches, app binary decryption/download, etc. (via ShmooCon)
- Twitter Infrastructure — an interview with Raffi Krikorian, VP of Platform Engineering. Details on SOA, deployment schedule, rollouts, and culture. (via Nelson Minar)
- Chromium is the New C Runtime — using Chrome’s open source core as the standard stack of networking, crash report, testing, logging, strings, encryption, concurrency, etc. libraries for C programming.
Learning Machine Learning, Pokemon Coding, Drone Coverage, and Optimization Guide
- CalTech Machine Learning Video Library — a pile of video introductions to different machine learning concepts.
- Awesome Pokemon Hack — each inventory item has a number associated with it, they are kept at a particular memory location, and there’s a glitch in the game that executes code at that location so … you can program by assembling items and then triggering the glitch. SO COOL.
- Drone Footage of Bangkok Protests — including water cannons.
- The Mature Optimization Handbook — free, well thought out, and well written. My favourite line: In exchange for that saved space, you have created a hidden dependency on clairvoyance.
A Free Velocity Report
When I first started as a sysadmin many years ago, I quickly realized what a daunting task was before me. Like any good engineer, I took to finding the right tools to keep at hand to make light work out of the most difficult situations. This in itself was quite an endeavor, as over the years there has been a proliferation of tools and scripts. Many are of the artisanal, organic, hand-crafted variety, forged out of bash pipelines by our forefathers.
Much of that has changed now as the DevOps movement strengthens. With closer interaction between developers and operations, or even operations teams composed of developers, the tools have significantly improved. Treating infrastructure as code with automated configuration management and provisioning tools have freed many from the menial tasks of creating snowflake systems, and we’ve turned our attention to the more important matters of scaling and optimizing our systems.
In my Velocity report, 5 Unsung Tools of DevOps, I highlight a few of the tools that have gone unnoticed—or at least unrecognized—for some time. These are but a few of the tools that recognized needs early on and that successfully solve real-world problems that you’re likely to encounter today. Here is a brief synopsis:
Help Searching, Offline First, AWS Tips, and Awesome Fonts
- Learn to Search — cheeky but spot-on help for people running conferences.
- Offline First — no, the mobile connectivity/bandwidth issue isn’t just going to solve itself on a global level anywhere in the near future. THIS!
- 10 Things You Should Know About AWS — lots of specialist tips for hardcore AWS users.
- The League of Moveable Type — AWESOME FONTS. Me gusta.
Time Series Database, Cluster Schedulers, Structural Search-and-Replace, and TV Data
- Influx DB — open-source, distributed, time series, events, and metrics database with no external dependencies.
- Omega (PDF) — ﬂexible, scalable schedulers for large compute clusters. From Google Research.
- Amazon Mines Its Data Trove To Bet on TV’s Next Hit (WSJ) — Amazon produced about 20 pages of data detailing, among other things, how much a pilot was viewed, how many users gave it a 5-star rating and how many shared it with friends.
Velocity 2013 Speaker Series
I want to start by thanking John and Steve for the warm welcome. They’ve created something very amazing with Velocity, and I’m excited to be a part of it.
It might seem a bit odd to talk about What’s Next at the beginning of a conference, but I figure the best time to go to the bank and ask for a loan is when you actually have some money.
What we’ve been talking about at Velocity, especially the DevOps side of things, is only the tip of the iceberg when it comes to how businesses are changing. And that shift is from the sequential to the concurrent. It used to be that we threw things over a series of walls, from Product Management to Design, to Development, to QA, to Production, to Customer Service and so on. That was an old world of software and one-year development cycles.
How you handle failure can mean the difference between "just another incident" and a revenue-stealing accident.
I was ready to get home. I’d been dozing throughout the flight from JFK to SFO, listening to the background chatter of Channel 9 as a lullaby. Somewhere over Sacramento, the rhythmic flow of controller-issued clearances and pilot confirmations was broken up by a call from our plane:
“NorCal Approach, United three-eighty-nine.”
“United three-eighty-nine, NorCal, go.”
“NorCal, United three-eighty-nine, we’d like to go ahead and…”
My headphones went silent, Channel 9 shut off.
I didn’t think too much of it as we continued our descent, flight attendants walking calmly through the cabin, getting us ready for landing. I had noticed our arrival path was one I was unfamiliar with, but nothing else seemed out of the ordinary… until we turned onto the final approach. In the turn, I noticed the unmistakable glint of firetrucks’ rotating red lights, lined up alongside the runway.
Explicit expectations are key to operating at scale
Our lives are rife with expectations.
When we flip the light switch, we expect electrons to flow; when we issue CPU instructions, we expect to get the correct answer; when we look at commit logs in the source repository, we (hopefully) expect tests to accompany them and that our colleagues have run them, pre-checkin. But we’ve all probably been burned by these types of assumptions at some point.
In an operational environment, like the large scale websites and build farms we’re responsible for, these sort of expectations can be a costly cause of errors, and are one of the prime sources of miscommunication. Many a postmortem has uncovered that some expectation the ops team had of the development team was actually an assumption… and we all know that old saying about assumptions and donkeys.
In the operational environment, miscommunication can be costly; but there are some easy ways to improve it.
Editor’s note: This is part two in a four-part series on the “-ations” of aviation that can provide further insight into DevOps best practices and achieving them. Part one, on how standardization helps organizations scale and is actually a part of healthy DevOps culture, can be read here.
Communication is an enigmatic topic when it comes to engineering. Parts of our jobs—blueprints, chemical formulae, and source code—require extremely precise forms of communication (even if it doesn’t end up communicating to the steel, molecules, or silicon what we intended). But when it comes to email threads sifting through requirements, meetings about implementation styles and risk assessment, and software design documentation, we often fumble.
Let’s face it: there’s a reason the “engineer equals bad communicator” stereotype exists. But there are some simple things that can be done, both individually and technologically, to begin challenging that stereotype.
Dual Navigation Receivers Required
There are obviously many forms of communication. In an operational context, it’s useful to distinguish between static and active communication.