"Velocity" entries

Swarm v. Fleet v. Kubernetes v. Mesos

Comparing different orchestration tools.

Buy Using Docker Early Release.

Buy Using Docker Early Release.

Most software systems evolve over time. New features are added and old ones pruned. Fluctuating user demand means an efficient system must be able to quickly scale resources up and down. Demands for near zero-downtime require automatic fail-over to pre-provisioned back-up systems, normally in a separate data centre or region.

On top of this, organizations often have multiple such systems to run, or need to run occasional tasks such as data-mining that are separate from the main system, but require significant resources or talk to the existing system.

When using multiple resources, it is important to make sure they are efficiently used — not sitting idle — but can still cope with spikes in demand. Balancing cost-effectiveness against the ability to quickly scale is difficult task that can be approached in a variety of ways.

All of this means that the running of a non-trivial system is full of administrative tasks and challenges, the complexity of which should not be underestimated. It quickly becomes impossible to look after machines on an individual level; rather than patching and updating machines one-by-one they must be treated identically. When a machine develops a problem it should be destroyed and replaced, rather than nursed back to health.

Various software tools and solutions exist to help with these challenges. Let’s focus on orchestration tools, which help make all the pieces work together, working with the cluster to start containers on appropriate hosts and connect them together. Along the way, we’ll consider scaling and automatic failover, which are important features.

Read more…

Comments: 18
Four short links: 15 October 2015

Four short links: 15 October 2015

The Chinese Dream, Siri Hacked, Indirect Measures, and Boring Technology

  1. Little Rice: Smartphones, Xiaomi, and the Chinese Dream (Amazon) — Clay Shirky’s new 128-page book/report about how Xiaomi exemplifies the balancing act that China has to perfect to navigate between cheap copies and innovation, between the demands of local and global markets, and between freedom and control. I’d buy Clay’s shopping list, the same way I’d gladly listen to Neil Gaiman telling the time. (via BoingBoing)
  2. Feed Siri Instructions From 16 Feet Away (Wired) — summary of a paywalled IEEE research paper Their clever hack uses those headphones’ cord as an antenna, exploiting its wire to convert surreptitious electromagnetic waves into electrical signals that appear to the phone’s operating system to be audio coming from the user’s microphone. […] It generates its electromagnetic waves with a laptop running the open source software GNU Radio, a USRP software-defined radio, an amplifier, and an antenna.
  3. User-Centered Design (Courtney Johnston) — the wall label should always give you cause to look back at the art work again. I love behaviour-based indirect measures of success like this.
  4. Choose Boring Technology (Dan McKinley) — going into the new hire required reading pile. See also the annotated slide deck.

Four short links: 14 October 2015

Four short links: 14 October 2015

Diversity Planning, Women in Robotics, AWS Resources, and Web Authentication

  1. Signals from Velocity New York “If your company is creating a diversity plan and you’ve actually gone and counted people,” Liles said, “you’ve already lost.” If you’re motivated to count, then know you’ve already lost. You want to know by how much.
  2. 25 Women in Robotics You Need to Know AboutThe DARPA Robotics Challenge (DRC) Finals 2015 were similarly lacking; of the 444 robot builders representing 24 robot entrants, only 23 builders were women (though some of the most successful teams at the DRC had female team members). Given how multidisciplinary the field is, and how many different skills are required, we need to celebrate women who are achieving greatness in robotics until we are seeing more parity. Great list.
  3. Awesome AWSA curated list of awesome Amazon Web Services (AWS) libraries, open source repos, guides, blogs, and other resources.
  4. The Web Authentication Arms RaceCryptography can only be used to transfer existing trust or secrecy across time or space; if the attacker impersonates the defender before the user establishes anything, it becomes impossible for the user to tell which party is legitimate. This sentence, made in solid gold Yes.

Signals from the 2015 O’Reilly Velocity Conference in New York

Key insights from DevOps, Web operations, and performance.

People from across the Web operations and performance worlds are coming together this week for the 2015 O’Reilly Velocity Conference in New York. Below, we’ve assembled notable keynotes, interviews, and insights from the event.

Diversity and bias

Bryan Liles, who works on strategic initiatives for DigitalOcean, gave a great thought-provoking talk on bias and diversity. “If your company is creating a diversity plan and you’ve actually gone and counted people,” Liles said, “you’ve already lost.”

Read more…

Four short links: 13 October 2015

Four short links: 13 October 2015

Apple Chips, Death of the Data Center, IBM R&D, and Stateful Services

  1. Apple’s Incredible Platform Advantage (Steve Cheney) — the best people in chip design no longer want to work at Intel or Qualcomm. They want to work at Apple. I have plenty of friends in the Valley who affirm this. Sure Apple products are cooler. But Apple has also surpassed Intel in performance. This is insane. A device company – which makes CPUs for internal use – surpassing Intel, the world’s largest chip maker that practically invented the CPU and has thousands of customers.
  2. Data Center’s Days are Numbered — Adrian Cockroft says, the investments going into bolstering security on AWS and other clouds are set to pay off to the point where within five years, “it will be impossible to get security certification if you’re not running in the cloud because the tools designed for data centers are sloppily put together and can’t offer the auditing for PCI and other regulators.”
  3. A Peek Inside IBM’s R&D LabIBM still has a physics department, but at this point, almost every physicist is somehow linked to a product plan or customer plan.
  4. Building Scalable Stateful Services (High Scalability) — elucidation of a talk by Caitie McCaffrey (YouTube), tech lead for observability at Twitter.

A “have-coffee” culture

Tending the DevOps victory garden.

Download a free copy of Building an Optimized Business, a curated collection of chapters from the O’Reilly Web Operations and Performance library. This post is an excerpt by J. Paul Reed from DevOps in Practice, one of the selections included in the curated collection.

Any discussion surrounding DevOps and its methodologies quickly comes to the often delicate issue of organizational dynamics and culture, at least if it’s an accurate treatment of the topic. There is often a tendency to downplay or gloss over these issues precisely because culture is thought of as a “squishy” thing, difficult to shape and change, and in some cases, to even address directly. But it doesn’t need to be this way.

Sam Hogenson, Vice President of Technology at Nordstrom, works hard to make sure it’s exactly the opposite: “At Nordstrom, we value these different experiences and we value the core of how you work, how you build relationships much more than whether or not you have subject matter expertise. It’s a successful formula.” Another part of that formula, Hogenson notes, is the ethos of the organization: “It’s a very empowered workforce, a very decentralized organization; I always remember the Nordstroms telling us ‘Treat this as if it were your name over the door: how would you run your business and take care of your customers?'” [Nordstrom infrastructure engineer Doug] Ireton described it as a “have-coffee culture: if you need to talk to someone, you go have coffee with them.”

Read more…


How we got to the HTTP/2 and HPACK RFCs

A brief history of SPDY and HTTP/2.

Download HTTP/2.

Download HTTP/2.

Download a free copy of HTTP/2. This post is an excerpt by Ilya Grigorik from High Performance Browser Networking, the essential guide to networking and web performance for web developers.

SPDY was an experimental protocol, developed at Google and announced in mid-2009, whose primary goal was to try to reduce the load latency of web pages by addressing some of the well-known performance limitations of HTTP/1.1. Specifically, the outlined project goals were set as follows:

  • Target a 50% reduction in page load time (PLT).
  • Avoid the need for any changes to content by website authors.
  • Minimize deployment complexity, avoid changes in network infrastructure.
  • Develop this new protocol in partnership with the open-source community.
  • Gather real performance data to (in)validate the experimental protocol.

To achieve the 50% PLT improvement, SPDY aimed to make more efficient use of the underlying TCP connection by introducing a new binary framing layer to enable request and response multiplexing, prioritization, and header compression.

Not long after the initial announcement, Mike Belshe and Roberto Peon, both software engineers at Google, shared their first results, documentation, and source code for the experimental implementation of the new SPDY protocol:

So far we have only tested SPDY in lab conditions. The initial results are very encouraging: when we download the top 25 websites over simulated home network connections, we see a significant improvement in performance—pages loaded up to 55% faster.

— A 2x Faster Web Chromium Blog

Fast-forward to 2012 and the new experimental protocol was supported in Chrome, Firefox, and Opera, and a rapidly growing number of sites, both large (e.g. Google, Twitter, Facebook) and small, were deploying SPDY within their infrastructure. In effect, SPDY was on track to become a de facto standard through growing industry adoption.

Read more…


Ghosts in the machines

The secret to successful infrastructure automation is people.


“The trouble with automation is that it often gives us what we don’t need at the cost of what we do.” —Nicholas Carr, The Glass Cage: Automation and Us

Virtualization and cloud hosting platforms have pervasively decoupled infrastructure from its underlying hardware over the past decade. This has led to a massive shift towards what many are calling dynamic infrastructure, wherein infrastructure and the tools and services used to manage it are treated as code, allowing operations teams to adopt software approaches that have dramatically changed how they operate. But with automation comes a great deal of fear, uncertainty and doubt.

Common (mis)perceptions of automation tend to pop up at the extreme ends: It will either liberate your people to never have to worry about mundane tasks and details, running intelligently in the background, or it will make SysAdmins irrelevant and eventually replace all IT jobs (and beyond). Of course, the truth is very much somewhere in between, and relies on a fundamental rethinking of the relationship between humans and automation.
Read more…

Comment: 1

The two-sided coin of Web performance

Hacking performance across your organization.

I’ve given Web performance talks where I get to show one of my favorite slides with the impact of third-party dependencies on load time. It’s the perfect use case for “those marketing people,” who overload pages with the tracking pixels and tags that make page load time go south. This, of course, would fuel the late-night pub discussrion with fellow engineers about how much faster the Web would be if those marketing people would attend a basic Web performance 101 course.

I’ve also found myself discussing exactly this topic in a meeting. This time, however, I was the guy arguing to keep the tracking code, although I was well aware of the performance impact. So what happened?
Read more…


How to leverage the browser cache with a CDN

An introduction to multi-level caching.


Since a content delivery network (CDN) is essentially a cache, you might be tempted not to make use of the cache in the browser, to avoid complexity. However, each cache has its own advantages that the other does not provide. In this post I will explain what the advantages of each are, and how to combine the two for the most optimal performance of your website.

Why use both?

While CDNs do a good job of delivering assets very quickly, they can’t do much about users who are out in the boonies and barely have a single bar of reception on their phone. As a matter of fact, in the US, the 95th percentile for the round trip time (RTT) to all CDNs is well in excess of 200 milliseconds, according to Cedexis reports. That means at least 5% of your users, if not more, are likely to have a slow experience with your website or application. For reference, the 50th percentile, or median, RTT is around 45 milliseconds.

So why bother using a CDN at all? Why not just rely on the browser cache?

Read more…