"devops" entries

7 takeaways from Velocity Europe

Taking a look at the current issues affecting the Web operations and performance space.

Editor’s note: The European edition of our Velocity conference wrapped up a few weeks ago, and now that the jet lag has passed I’ve had a chance to reflect on the talks and excellent hallway conversations I had throughout. And while I thoroughly enjoyed all the sessions I introduced, one of the downsides to being a chair is that I can’t attend all the other sessions at the same time. As such, I always look around for excellent dissections of the conference from other people; this summary by Peter Arijs from CoScale closely reflects some of the themes I saw, including a few of the standout talks.

velocity_barcelona_crop

November in Barcelona was full of action for web and big data practitioners, with the Velocity and Strata-Hadoop conferences and side events such as WebPerfDays and Papis.io. As a startup in the web application monitoring and analytics space, it was the perfect time to get a pulse on the state of the art, and talk to some of our clients and prospects. Below is a summary of personal take-away points from selected Velocity sessions and personal interactions.

Read more…

Comment
Four short links: 15 December 2014

Four short links: 15 December 2014

Transferable Learning, At-Scale Telemetry, Ugly DRM, and Fast Packet Processing

  1. How Transferable Are Features in Deep Neural Networks? — (answer: “very”). A final surprising result is that initializing a network with transferred features from almost any number of layers can produce a boost to generalization that lingers even after fine-tuning to the target dataset. (via Pete Warden)
  2. Introducing Atlas: Netflix’s Primary Telemetry Platform — nice solution to the problems that many have, at a scale that few have.
  3. The Many Facades of DRM (PDF) — Modular software systems are designed to be broken into independent pieces. Each piece has a clear boundary and well-defined interface for ‘hooking’ into other pieces. Progress in most technologies accelerates once systems have achieved this state. But clear boundaries and well-defined interfaces also make a technology easier to attack, break, and reverse-engineer. Well-designed DRMs have very fuzzy boundaries and are designed to have very non-standard interfaces. The examples of the uglified DRM code are inspiring.
  4. DPDKa set of libraries and drivers for fast packet processing […] to: receive and send packets within the minimum number of CPU cycles (usually less than 80 cycles); develop fast packet capture algorithms (tcpdump-like); run third-party fast path stacks.
Comment
Four short links: 8 December 2014

Four short links: 8 December 2014

Systemic Improvement, Chinese Trends, Deep Learning, and Technical Debt

  1. Reith Lectures — this year’s lectures are by Atul Gawande, talking about preventable failure and systemic improvement — topics of particular relevance to devops cultural devotees. (via BoingBoing)
  2. Chinese Mobile App UI Trends — interesting differences between US and China. Phone number authentication interested me: You key in your number and receive a confirmation code via SMS. Here, all apps offer this type of phone number registration/login (if not prefer it). This also applies to websites, even those without apps. (via Matt Webb)
  3. Large Scale Deep Learning (PDF) — Jeff Dean from Google. Starts easy! Starts.
  4. Machine Learning: The High-Interest Credit Card of Technical Debt (PDF) — Google research paper on the ways in which machine learning can create problems rather than solve them.
Comment: 1
Four short links: 28 November 2014

Four short links: 28 November 2014

Material Design Inspiration, Event Processing, Launch Infrastructure, Remote Work

  1. Material Up — material design inspiration. MD is a physics engine for UI.
  2. Flafka (Cloudera) — Flume plus Kafka, offers sub-second-latency event processing without the need for dedicated infrastructure. (via Abishek Tiwari)
  3. terraform.io — open source package providing a common configuration to launch infrastructure, from physical and virtual servers to email and DNS providers.
  4. Remote Work: An Engineering Leader’s PerspectiveEven proponents of remote work seem to think that you should either have a distributed team from the get go, or stick to a traditional on-site team. Our experience shows that this is incorrect…
Comment
Four short links: 20 November 2014

Four short links: 20 November 2014

Postmortems, Cloud Triggers, IoT Desires, and Barbie Can Code

  1. The Infinite Hows (John Allspaw) — when finding ways to improve systems to prevent errors, the process of diagnosis should be focused on the systems and less on the people. (aka “human error” is the result of a preceding systems error.) (aka “design for failure.”)
  2. Amazon Lambda — triggers in the cloud.
  3. Enchanted Objects (PNG) — organizing the Internet of Thing by human desires. (via Designing the Enchanted Future)
  4. Barbie Remixed (PDF) — brilliant remix of a book that missed the mark into one that hits the bullseye.
Comment
Four short links: 19 November 2014

Four short links: 19 November 2014

Current Software Practices, Future Science Practices, Javascript Typechecking, and Microservices for Scala

  1. Distributed Developer Stack Field Guide (O’Reilly) — making sense of what software development and deployment now looks like. (via O’Reilly Radar)
  2. Data Capture for the Real World (Cameron Neylon) — there’s a huge opportunity for science IT: tracking data as scientists do their work, and then with massive audit trails and provenance info. Think Salesforce for experiments.
  3. Flow — static type checking for Javascript, from Facebook.
  4. ColossusI/O and Microservice library for Scala from Tumblr engineering.
Comment

Introducing “A Field Guide to the Distributed Development Stack”

Tools to develop massively distributed applications.

Editor’s Note: At the Velocity Conference in Barcelona we launched “A Field Guide to the Distributed Development Stack.” Early response has been encouraging, with reactions ranging from “If I only had this two years ago” to “I want to give a copy of this to everyone on my team.” Below, Andrew Odewahn explains how the Guide came to be and where it goes from here.


As we developed Atlas, O’Reilly’s next-generation publishing tool, it seemed like every day we were finding interesting new tools in the DevOps space, so I started a “Sticky” for the most interesting-looking tools to explore.

field-guide-sticky

At first, this worked fine. I was content to simply keep a list, where my only ordering criteria was “Huh, that looks cool. Someday when I have time, I’ll take a look at that,” in the same way you might buy an exercise DVD and then only occasionally pull it out and think “Huh, someday I’ll get to that.” But, as anyone who has watched DevOps for any length of time can tell you, it’s a space bursting with interesting and exciting new tools, so my list and guilt quickly got out of hand.

Read more…

Comment
Four short links: 18 November 2014

Four short links: 18 November 2014

A Worm Mind Forever LEGO Voyaging, Automatic Caption Generator, ELK Stack, and Amazonian Deployment

  1. A Worm’s Mind in a Lego Body — the c. elegans worm’s 302 neurons has been sequenced, modelled in open source code, and now hooked up to a Lego robot. It is claimed that the robot behaved in ways that are similar to observed C. elegans. Stimulation of the nose stopped forward motion. Touching the anterior and posterior touch sensors made the robot move forward and back accordingly. Stimulating the food sensor made the robot move forward. There is video.
  2. Show and Tell: A Neural Image Caption Generator — Google Research paper on generating captions like “Two pizzas sitting on top of a stove top oven” from a photo. Wow.
  3. Big Data with the ELK Stack — ElasticSearch, logstash, and Kibana. Interesting and powerful combination of tools!
  4. Apollo: Amazon’s Deployment EngineApollo will stripe the rolling update to simultaneously deploy to an equivalent number of hosts in each location. This keeps the fleet balanced and maximizes redundancy in the case of any unexpected events. When the fleet scales up to handle higher load, Apollo automatically installs the latest version of the software on the newly added hosts. Lust.
Comment
Four short links: 17 November 2014

Four short links: 17 November 2014

Tut Tut ISPs, Distributing Old Datastores, Secure Containers, and Design Workflow

  1. ISPs Remove Their Customers’ Email Encryption (EFF) — ISPs have apparently realised that man-in-the-middle is their business model.
  2. Dynomite (Netflix) — a sharding and replication layer. Dynomite can make existing non-distributed datastores, such as Redis or Memcached, into a fully distributed & multi-datacenter replicating datastore.
  3. After Dockersmaller, easier to manage, more secure containers via unikernels and immutable infrastructure.
  4. Pixelapse — something between Dropbox and Github for the design workflow and artifacts.
Comment: 1
Four short links: 14 November 2014

Four short links: 14 November 2014

Completing Maps, ChatOps, Career Design, and Data Privacy

  1. Missing Maps Fill In the Blanks (New Scientist) — OpenStreetMap project to crowdmap slums around the world.
  2. Chatops — devops deployment chatter with Hubot.
  3. Alternatives to Tech Career Ladders — Spotify trying to figure out how to keep engineers challenged as they become more senior.
  4. Mozilla’s Data Privacy Principles — well-articulated and useful: without pre-defined principles, it’s so easy to accidentally collect or poorly protect data.
Comment