- The Virtuous Pipeline of Code (Public Resource) — Chicago partnering with Public Resource to open its legal codes for good. “This is great! What can we do to help?” Bravo Chicago, and everyone else—take note!
- Smithsonian’s 3D Data — models of 21 objects, from a gunboat to the Wright Brothers’ plane, to a wooly mammoth skeleton, to Lincoln’s life masks. I wasn’t able to find a rights statement on the site which explicitly governed the 3D models. (via Smithsonian Magazine)
- Anki’s Robot Cars (Xconomy) — The common characteristics of these future products, in Sofman’s mind: “Relatively simple and elegant hardware; incredibly complicated software; and Web and wireless connectivity to be able to continually expand the experience over time.” (via Slashdot)
- An Empirical Evaluation of TCP Performance in Online Games — We show that because TCP was originally designed for unidirectional and network-limited bulk data transfers, it cannot adapt well to MMORPG traffic. In particular, the window-based congestion control and the fast retransmit algorithm for loss recovery are ineffective. Furthermore, TCP is overkill, as not every game packet needs to be transmitted in a reliably and orderly manner. We also show that the degraded network performance did impact users’ willingness to continue a game.
"Velocity" entries
Four short links: 18 November 2013
Chicago Code, 3D Smithsonian Data, AI Controlling Everything, and Game TCP
Velocity NY recap
Operating on the edge and real-time performance emerged as key themes at Velocity NY.
Maybe it was the hustle and bustle of Times Square just within earshot. Maybe it was the smell of that legendary pizza on every corner. Or maybe it was having a front row seat in the financial capital of the world while the drama of a possible government default played itself out like a Broadway show. Whatever it was, Velocity New York felt markedly different than its west coast cousin.
Of course, Velocity’s inaugural east coast incarnation sported presentations on the state of the art of all the topics we’ve come to expect from the conference: nudging style sheets, hacking Javascript, battling memory usage, tuning database queries, and coaxing the network for every last ounce of performance on the Web. (And then doing it all over again for the mobile version…)
But discussions on scaling the operations of the squishy side of the organization–developing healthy team interactions, talking through the realities of the ever-elusive DevOps culture, and what it all means to scaling not just web sites but full-fledged businesses–were more present than ever.
Velocity: Toward the real-time business
Velocity 2013 Speaker Series
I want to start by thanking John and Steve for the warm welcome. They’ve created something very amazing with Velocity, and I’m excited to be a part of it.
It might seem a bit odd to talk about What’s Next at the beginning of a conference, but I figure the best time to go to the bank and ask for a loan is when you actually have some money.
What we’ve been talking about at Velocity, especially the DevOps side of things, is only the tip of the iceberg when it comes to how businesses are changing. And that shift is from the sequential to the concurrent. It used to be that we threw things over a series of walls, from Product Management to Design, to Development, to QA, to Production, to Customer Service and so on. That was an old world of software and one-year development cycles.
Building for failure is a recipe for success
How you handle failure can mean the difference between "just another incident" and a revenue-stealing accident.
I was ready to get home. I’d been dozing throughout the flight from JFK to SFO, listening to the background chatter of Channel 9 as a lullaby. Somewhere over Sacramento, the rhythmic flow of controller-issued clearances and pilot confirmations was broken up by a call from our plane:
“NorCal Approach, United three-eighty-nine.”
“United three-eighty-nine, NorCal, go.”
“NorCal, United three-eighty-nine, we’d like to go ahead and…”
My headphones went silent, Channel 9 shut off.
I didn’t think too much of it as we continued our descent, flight attendants walking calmly through the cabin, getting us ready for landing. I had noticed our arrival path was one I was unfamiliar with, but nothing else seemed out of the ordinary… until we turned onto the final approach. In the turn, I noticed the unmistakable glint of firetrucks’ rotating red lights, lined up alongside the runway.
Going beyond Onload: Measuring performance that matters
Velocity 2013 Speaker Series: Focus on Web Apps, Not Web Pages
We’re not making web pages anymore; we’re building web applications. Gone are the days of a few script tags in the <head>. Apps today are a complex web of asynchronously-loaded content and functionality. In the past decade, we’ve progressed from statically-loaded HTML to AJAX-ifying all the things. However, the way we’ve been measuring real user performance of our apps hasn’t changed to reflect our new state of art.
Defining “Done”
At what point during page load do users consider an app to be “ready enough” to start using? If we use standard performance metrics, we have to choose one of the following:
1) When the HTML document has been completely loaded and parsed, but before stylesheets, images, and subframes have finished loading (DOMContentLoaded)
2) When all synchronous scripts, stylesheets, images, and subframes have finished loading (onload)
If we pick DOMContentLoaded, it quickly becomes clear that there’s no inherent correlation between the app state at that point and what a user would consider “ready.”
Measuring Mobile Performance
Velocity 2013 Speaker Series: A Sneak Peek With WebPagetest and Appurify
Now that more companies have basic mobile strategies in place, they are turning their attention to the issue of performance.
Mobile developers are thinking about how fast their apps and mobile webpages load and—more importantly—what they can do to make them faster. Consumers have little patience for slow loading apps and their expectations are only going to get more stringent. This expectation likely contributed to Apple making changes so that apps on iOS 7 load 11% faster than on iOS 6.
The challenge is that measuring performance for mobile is not as easy as it is for web. Many of us have used tools like WebPagetest to assess website performance across different browsers/locations and pinpoint areas for improvement but fully functional, equivalent tools don’t exist yet for the mobile space.
This has left mobile developers ill equipped to create the highest-performing mobile apps and websites.
Read more…
Making Systems Operable
Velocity 2013 Speaker Series
There’s an old joke about the aviation cockpit of the future that it will contain just a pilot and a dog. The pilot will be there to watch the automation. The dog will be there to bite the pilot if he tries to touch anything.
Although they will all deny it, the majority of modern IT developers have exactly this view of automation: the system is designed to be self regulating and operators are there to watch it, not to operate it. The result is current systems are often inoperable, i.e. systems they cannot be effectively operated because their functions and capacities are hidden or inaccessible.
The conceit in the pilot-and-the-dog joke is that modern systems do not require operation, that they are autonomous. Whenever these systems are exhibited, our attention is drawn to their autonomous features. But there are no systems that actually function without operators. Even when we claim they are “unmanned”, all important systems have operators who are intimately involved in their function: UAV’s are piloted, the Mars rover is driven, the satellites are managed, surgical robots are manipulated, insulin pumps are programmed. We do not see these activities–many are performed by workers who remain anonymous–but we depend on them.
The Joys of Static Memory JavaScript
Velocity 2013 Speaker Series
You wake up one morning to discover your team has gotten a dreaded alert: your web application is performing badly. You dig through your code, but don’t see anything that stands out, until you open up Chrome’s memory performance tools, and see this:
One of your co-workers chuckles, because they realize that you’ve got a memory-related performance problem.
Building an Alerting System That Really Works
Velocity 2013 Speaker Series
Building a high quality alerting system often feels like a dark art. Often it is hard to set the proper thresholds and it is even harder to define when an alert should be triggered or not. This results in alerts being raised too early or too late and your colleagues losing faith in the system. Once you use a structured approach to build an alerting system you will find it much easier and the alerts more predictable and precise.
Measure Selection
First you have to select proper measures to alert on. This selection is key as all other steps depend on using meaningful measures. While there seems to be an infinite number of different measures, you can categorize them into three main categories:
- Saturation measures indicate how much of a resource is used. Examples are CPU usage or resource pool consumption.
- Occurrence measures indicate whether a condition was met or not. A good example is errors. These measures are often presented as a rate like failed transactions per seconds.
- Continuous measures do not have a single value at any given point in time, but instead a large number of different values. A typical example is response times. Irrespective of how small you make the sample, you will always have a large amount of values and never just one single representative value.
Sharing is a competitive advantage
Why the Velocity conference is coming to New York.
In October, we’re bringing our Velocity conference to New York for the first time. Let’s face it, a company expanding its conference to other locations isn’t anything that unique. And given the thriving startup scene in New York, there’s no real surprise we’d like to have a presence there, either. In that sense, we’ll be doing what we’ve already been doing for years with the Velocity conference in California: sharing expert knowledge about the skills and technologies that are critical for building scalable, resilient, high-availability websites and services.
But there’s an even more compelling reason we’re looking to New York: the finance industry. We’d be foolish and remiss if we acted like it didn’t factor in to our decision, and that we didn’t also share some common concerns, especially on the operational side of things. The Velocity community spends a great deal of time navigating significant operational realities — infrastructure, cost, risk, failures, resiliency; we have a great deal to share with people working in finance, and I’d wager, a great deal to learn in return. If Google or Amazon go down, they lose money. (I’m not saying this is a good thing, mind you.) When a “technical glitch” occurs in financial service systems, we get flash crashes, a complete suspension of the Nasdaq, and whatever else comes next — all with potentially catastrophic outcomes. Read more…