- Interacting with a World of Connected Objects (Tom Coates) — notes from one of my favourite Foo Camp sessions.
- Security Considerations with Continuous Deployment (IBM) — rundown of categories of security issues your org might face, and how to tackle them in the continuous deployment cycle. (via Emma Jane Westby)
- The Chess Master and the Computer (Garry Kasparov) — Increasingly, a move isn’t good or bad because it looks that way or because it hasn’t been done that way before. It’s simply good if it works and bad if it doesn’t. Although we still require a strong measure of intuition and logic to play well, humans today are starting to play more like computers. (via Alexis Madrigal)
- Virtual Machines, Javascript, and Assembler (YouTube) — hilarious Velocity keynote by Scott Hanselman.
"Velocity" entries
Four short links: 30 June 2014
Interacting with Connected Objects, Continuous Security Review, Chess AI, and Scott Hanselman is Hilarious
Four short links: 27 June 2014
Google MillWheel, 20yo Bug, Fast Real-Time Visualizations, and Google's Speed King
- MillWheel: Fault-Tolerant Stream Processing at Internet Scale — Google Research paper on the tech underlying the new cloud DataFlow tool. Watch the video. Yow.
- The Integer Overflow Bug That Went to Mars — long-standing (20 year old!) bug in a compression library prompts a wave of new releases. No word yet on whether NASA will upgrade the rover to avoid being pwned by Martian script kiddies. (update: I fell for a self-promoter. The Martians will need to find another attack vector. Huzzah!)
- epoch (github) — Fastly-produced open source general purpose real-time charting library for building beautiful, smooth, and high performance visualizations.
- Achieving Rapid Response Times in Large Online Services (YouTube) — Jeff Dean‘s keynote at Velocity. He wrote … a lot of things for this. And now he’s into deep learning ….
Four short links: 26 June 2014
IoT Future, Latency Numbers, Mobile Performance, and Minimum Viable Bureaucracy
- Charlie Stross on 2034 — every object in the real world is going to be providing a constant stream of metadata about its environment — and I mean every object. The frameworks used for channeling this firehose of environment data are going to be insecure and ramshackle, with foundations built on decades-old design errors. (via BoingBoing)
- Latency Numbers Every Programmer Should Know — awesome animation so you can see how important “constants” which drive design decisions have changed over time.
- Extreme Web Performance for Mobile Devices (Slideshare) — notes from Maximiliano Firtman’s Velocity tutorial.
- Minimum Viable Bureaucracy (Laura Thomson) — notes from her Velocity talk. A portion of engineer’s time must be spent on what engineer thinks is important. It may be 100%. It may be 60%, 40%, 20%. But it should never be zero.
Four short links: 20 June 2014
Available Data, Goal Setting, Real Tech, and Gamification Numbers
- Dynamo and BigTable — good preso overview of two approaches to solving availability and consistency in the event of server failure or network partition.
- Goals Gone Wild (PDF) — In this article, we argue that the beneficial effects of goal setting have been overstated and that systematic harm caused by goal setting has been largely ignored. We identify specific side effects associated with goal setting, including a narrow focus that neglects non-goal areas, a rise in unethical behavior, distorted risk preferences, corrosion of organizational culture, and reduced intrinsic motivation.
- Tech Isn’t All Brogrammers (Alexis Madrigal) — a reminder that there are real scientists and engineers in Silicon Valley working on problems considerably harder than selling ads and delivering pet food to one another. (via Brian Behlendorf)
- Numbers from 90+ Gamification Case Studies — cherry-picked anecdata for your business cases.
From the network interface to the database
All systems are distributed systems, and we’re starting to see how they fit into Velocity's themes.
From the beginning, the Velocity Conference has focused on web performance and operations — specifically, web operations. This focus has been fairly narrow: browser performance dominated the discussion of “web performance,” and interactions between developers and IT staff dominated operations.
These limits weren’t bad. Perceived performance really is dominated by the browser — how fast you can get resources (HTML, images, CSS files, JavaScript libraries) over the network to the browser, and how fast the browser can execute those resources. How long before a user stops waiting for your page to load and clicks away? How do you make a page useable as quickly as possible, even before all the resources have loaded? Those discussions were groundbreaking and surprising: users are incredibly sensitive to page speed.
That’s not to say that Velocity hasn’t looked at the rest of the application stack; there’s been an occasional glance in the direction of the database and an even more occasional glance at the middleware. But the database and middleware have, at least historically, played a bit part. And while the focus of Velocity has been front-end tuning, speakers like Baron Schwartz haven’t let us ignore the database entirely. Read more…
Beyond the stack
The tools in the Distributed Developer's Stack make development manageable in a highly distributed environment.
The shape of software development has changed radically in the last two decades. We’ve seen many changes: the Internet, the web, virtualization, and cloud computing. All of these changes point toward a fundamental new reality: all computing has become distributed computing. The age of standalone applications has disappeared, and applications that run on a single computer are almost inconceivable. Distributed is the default; and whether an application is running on Amazon Web Services (AWS), on a private cloud, or even on a desktop or a mobile phone, it depends on the behavior of other systems and services that aren’t under the developer’s control.
In the past few years, a new toolset has grown up to support the development of massively distributed applications. We call this new toolset the Distributed Developer’s Stack (DDS). It is orthogonal to the more traditional world of servers, frameworks, and operating systems; it isn’t a replacement for the aged LAMP stack, but a set of tools to make development manageable in a highly distributed environment.
The DDS is more of a meta-stack than a “stack” in the traditional sense. It’s not prescriptive; we don’t care whether you use AWS or OpenStack, whether you use Git or Mercurial. We do care that you develop for the cloud, and that you use a distributed version control system. The DDS is about the requirements for working effectively in the second decade of the 21st century. The specific tools have evolved, and will continue to evolve, and we expect you to evolve, too. Read more…
Everything is distributed
How do we manage systems that are too large to understand, too complex to control, and that fail in unpredictable ways?
“What is surprising is not that there are so many accidents. It is that there are so few. The thing that amazes you is not that your system goes down sometimes, it’s that it is up at all.”—Richard Cook
In September 2007, Jean Bookout, 76, was driving her Toyota Camry down an unfamiliar road in Oklahoma, with her friend Barbara Schwarz seated next to her on the passenger side. Suddenly, the Camry began to accelerate on its own. Bookout tried hitting the brakes, applying the emergency brake, but the car continued to accelerate. The car eventually collided with an embankment, injuring Bookout and killing Schwarz. In a subsequent legal case, lawyers for Toyota pointed to the most common of culprits in these types of accidents: human error. “Sometimes people make mistakes while driving their cars,” one of the lawyers claimed. Bookout was older, the road was unfamiliar, these tragic things happen. Read more…
Four short links: 28 January 2014
Client-Server, Total Information Awareness, MSFT Joins OCP, and Tissue Modelling
- Intel On-Device Voice Recognition (Quartz) — interesting because the tension between client-side and server-side functionality is still alive and well. Features migrate from core to edge and back again as cycles, data, algorithms, and responsiveness expectations change.
- Meet Microsoft’s Personal Assistant (Bloomberg) — total information awareness assistant. By Seeing, Hearing, and Knowing All, in the future even elevators will be trying to read our minds. (via The Next Web)
- Microsoft Contributes Cloud Server Designs to Open Compute Project — As part of this effort, Microsoft Open Technologies Inc. is open sourcing the software code we created for the management of hardware operations, such as server diagnostics, power supply and fan control. We would like to help build an open source software community within OCP as well. (via Data Center Knowledge)
- Open Tissue Wiki — open source (ZLib license) generic algorithms and data structures for rapid development of interactive modeling and simulation.
Four short links: 3 January 2014
Mesh Networks, Collaborative LaTeX, Distributed Systems Book, and Reverse-Engineering Netflix Metadata
- Commotion — open source mesh networks.
- WriteLaTeX — online collaborative LaTeX editor. No, really. This exists. In 2014.
- Distributed Systems — free book for download, goal is to bring together the ideas behind many of the more recent distributed systems – systems such as Amazon’s Dynamo, Google’s BigTable and MapReduce, Apache’s Hadoop etc.
- How Netflix Reverse-Engineered Hollywood (The Atlantic) — Using large teams of people specially trained to watch movies, Netflix deconstructed Hollywood. They paid people to watch films and tag them with all kinds of metadata. This process is so sophisticated and precise that taggers receive a 36-page training document that teaches them how to rate movies on their sexually suggestive content, goriness, romance levels, and even narrative elements like plot conclusiveness.
Four short links: 19 November 2013
Ad Triumphalism, Education Not Transformed, Bookstore Infrastructure, and Tossable Camera
- Why The Banner Ad is Heroic — enough to make Dave Eggers cry. Advertising triumphalism rampant.
- Udacity/Thrun Profile — A student taking college algebra in person was 52% more likely to pass than one taking a Udacity class, making the $150 price tag–roughly one-third the normal in-state tuition–seem like something less than a bargain. In which Udacity pivots to hiring-sponsored workforce training and the new educational revolution looks remarkably like sponsored content.
- Amazon is Building Substations (GigaOm) — the company even has firmware engineers whose job it is to rewrite the archaic code that normally runs on the switchgear designed to control the flow of power to electricity infrastructure. Pretty sure that wasn’t a line item in the pitch deck for “the first Internet bookstore”.
- Panoramic Images — throw the camera in the air, get a 360×360 image from 36 2-megapixel lenses. Not sure that throwing was previously a recognised UI gesture.