"distributed computing" entries

Four short links: 18 November 2014

Four short links: 18 November 2014

A Worm Mind Forever LEGO Voyaging, Automatic Caption Generator, ELK Stack, and Amazonian Deployment

  1. A Worm’s Mind in a Lego Body — the c. elegans worm’s 302 neurons has been sequenced, modelled in open source code, and now hooked up to a Lego robot. It is claimed that the robot behaved in ways that are similar to observed C. elegans. Stimulation of the nose stopped forward motion. Touching the anterior and posterior touch sensors made the robot move forward and back accordingly. Stimulating the food sensor made the robot move forward. There is video.
  2. Show and Tell: A Neural Image Caption Generator — Google Research paper on generating captions like “Two pizzas sitting on top of a stove top oven” from a photo. Wow.
  3. Big Data with the ELK Stack — ElasticSearch, logstash, and Kibana. Interesting and powerful combination of tools!
  4. Apollo: Amazon’s Deployment EngineApollo will stripe the rolling update to simultaneously deploy to an equivalent number of hosts in each location. This keeps the fleet balanced and maximizes redundancy in the case of any unexpected events. When the fleet scales up to handle higher load, Apollo automatically installs the latest version of the software on the newly added hosts. Lust.

Small brains, big data

How neuroscience is benefiting from distributed computing — and how computing might learn from neuroscience.


When we think about big data, we usually think about the web: the billions of users of social media, the sensors on millions of mobile phones, the thousands of contributions to Wikipedia, and so forth. Due to recent innovations, web-scale data can now also come from a camera pointed at a small, but extremely complex object: the brain. New progress in distributed computing is changing how neuroscientists work with the resulting data — and may, in the process, change how we think about computation. Read more…

Comments: 3

Revisiting “What is DevOps”

If all companies are software companies, then all companies must learn to manage their online operations.


Two years ago, I wrote What is DevOps. Although that article was good for its time, our understanding of organizational behavior, and its relationship to the operation of complex systems, has grown.

A few themes have become apparent in the two years since that last article. They were latent in that article, I think, but now we’re in a position to call them out explicitly. It’s always easy to think of DevOps (or of any software industry paradigm) in terms of the tools you use; in particular, it’s very easy to think that if you use Chef or Puppet for automated configuration, Jenkins for continuous integration, and some cloud provider for on-demand server power, that you’re doing DevOps. But DevOps isn’t about tools; it’s about culture, and it extends far beyond the cubicles of developers and operators. As Jeff Sussna says in Empathy: The Essence of DevOps:

…it’s not about making developers and sysadmins report to the same VP. It’s not about automating all your configuration procedures. It’s not about tipping up a Jenkins server, or running your applications in the cloud, or releasing your code on Github. It’s not even about letting your developers deploy their code to a PaaS. The true essence of DevOps is empathy.

Read more…

Comments: 4