"infrastructure" entries

Four short links: 31 July 2015

Four short links: 31 July 2015

Robot Swarms, Google Datacenters, VR Ecosystem, and DeepDream Visualised

  1. Buzz: An Extensible Programming Language for Self-Organizing Heterogeneous Robot Swarms (arXiv) — Swarm-based primitives allow for the dynamic management of robot teams, and for sharing information globally across the swarm. Self-organization stems from the completely decentralized mechanisms upon which the Buzz run-time platform is based. The language can be extended to add new primitives (thus supporting heterogeneous robot swarms), and its run-time platform is designed to be laid on top of other frameworks, such as Robot Operating System.
  2. Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google’s Datacenter Network (PDF) — Our datacenter networks run at dozens of sites across the planet, scaling in capacity by 100x over 10 years to more than 1Pbps of bisection bandwidth. Wow, their Wi-Fi must be AMAZING!
  3. Nokia’s VR Ambitions Could Restore Its Tech Lustre (Bloomberg) — the VR ecosystem map is super-interesting.
  4. Visualising GoogleNet Classes — fascinating to see squirrel monkeys and basset hounds emerge from nothing. It’s so tempting to say, “this is what the machine sees in its mind when it thinks of basset hounds,” even though Boring Brain says, “that’s bollocks and you know it!”
Comment
Four short links: 27 July 2015

Four short links: 27 July 2015

Google’s Borg, Georgia v. Malamud, SLAM-aware system, and SmartGPA

  1. Large-scale Cluster Management at Google with BorgGoogle’s Borg system is a cluster manager that runs hundreds of thousands of jobs, from many thousands of different applications, across a number of clusters, each with up to tens of thousands of machines. […] We present a summary of the Borg system architecture and features, important design decisions, a quantitative analysis of some of its policy decisions, and a qualitative examination of lessons learned from a decade of operational experience with it.
  2. Georgia Sues Carl Malamud (TechDirt) — for copyright infringement… for publishing an official annotated copy of the state's laws. […] the state points directly to the annotated version as the official laws of the state.
  3. Monocular SLAM Supported Object Recognition (PDF) — a monocular SLAM-aware object recognition system that is able to achieve considerably stronger recognition performance, as compared to classical object recognition systems that function on a frame-by-frame basis. (via Improving Object Recognition for Robots)
  4. SmartGPA: How Smartphones Can Assess and Predict Academic Performance of College Students (PDF) — We show that there are a number of important behavioral factors automatically inferred from smartphones that significantly correlate with term and cumulative GPA, including time series analysis of activity, conversational interaction, mobility, class attendance, studying, and partying.
Comment: 1
Four short links: 15 May 2015

Four short links: 15 May 2015

Army Cloud, Google Curriculum, Immutable Infrastructure, and Task Queues

  1. Army Cloud Computing Strategy (PDF) — aka: “what we hope to do without having done, to use what we’re doing to them.”
  2. Guide to Technical Development (Google) — This guide is a suggested path for university students to develop their technical skills academically and non-academically through self-paced, hands-on learning.
  3. Immutable Infrastructure is the Future (Michael DeHaan) — The future of configuration management systems is in deploying cloud infrastructure that will later run immutable systems via an API level.
  4. machineryan asynchronous task queue/job queue based on distributed message passing.
Comment
Four short links: 12 May 2015

Four short links: 12 May 2015

Data Center Numbers, Utility Computing, NSA Art, and RIP CAP

  1. We Used to Build Steel Mills Near Cheap Sources of Power, but Now That’s Where We Build DatacentersHennessy & Patterson estimate that of the $90M cost of an example datacenter (just the facilities – not the servers), 82% is associated with power and cooling. The servers in the datacenter are estimated to only cost $70M. It’s not fair to compare those numbers directly since servers need to get replaced more often than datacenters; once you take into account the cost over the entire lifetime of the datacenter, the amortized cost of power and cooling comes out to be 33% of the total cost, when servers have a three-year lifetime and infrastructure has a 10-15 year lifetime. Going back to the Barroso and Holzle book, processors are responsible for about a third of the compute-related power draw in a datacenter (including networking), which means that just powering processors and their associated cooling and power distribution is about 11% of the total cost of operating a datacenter. By comparison, the cost of all networking equipment is 8%, and the cost of the employees that run the datacenter is 2%.
  2. Microsoft Invests in 3 Undersea Cable Projects — utility computing is an odd concept, given how quickly hardware cycles refresh. In the past, you could ask whether investors wanted to be in a high-growth, high-risk technology business or a stable blue-chip utility.
  3. Secret Power — Simon Denny’s NSA-logo-and-Snowden-inspired art makes me wish I could get to Venice. See also The Guardian piece on him.
  4. Please Stop Calling Databases CP or AP (Martin Kleppman) — The fact that we haven’t been able to classify even one datastore as unambiguously “AP” or “CP” should be telling us something: those are simply not the right labels to describe systems. I believe that we should stop putting datastores into the “AP” or “CP” buckets. So readable!
Comment
Four short links: 11 May 2015

Four short links: 11 May 2015

Age of Infrastructure, Facial Expressions, Proof Assistants, and Programmer Talent

  1. Welcome to the Age of Infrastructure (Annalee Newitz) — The Internet isn’t that thing in there, inside your little glowing box. It’s in your washing machine, kitchen appliances, pet feeder, your internal organs, your car, your streets, the very walls of your house. You use your wearable to interface with the world out there.
  2. Facial Performance Sensing Head-Mounted Display (YouTube) — glorious use of an Oculus headset, to capture (for reproduction on an avatar) fine-grained facial expressions. From SIGGRAPH 2015.
  3. Mathematical Proof Assistants — human augmentation in mathematics.
  4. The Programmer Talent Myth (LWN) — Jacob Kaplan-Moss on the distribution of programmer talent and the damage that the bimodal myth causes.
Comments: 3
Four short links: 27 January 2015

Four short links: 27 January 2015

Autonomous Corporations, Abstract Thought, Down Rounds, and Distributed Messaging

  1. Decentralised Autonomous Corporations — Charlie Stross’s near-future fiction of Accelerando comes closer to reality: Malice – revenge for waking him up – sharpens Manfred’s voice. “The president of agalmic.holdings.root.184.97.AB5 is agalmic.holdings.root.184.97.201. The secretary is agalmic.holdings.root.184.D5, and the chair is agalmic.holdings.root.184.E8.FF. All the shares are owned by those companies in equal measure, and I can tell you that their regulations are written in Python. Have a nice day, now!” He thumps the bedside phone control and sits up, yawning, then pushes the do-not-disturb button before it can interrupt again. After a moment he stands up and stretches, then heads to the bathroom to brush his teeth, comb his hair, and figure out where the lawsuit originated and how a human being managed to get far enough through his web of robot companies to bug him.
  2. Coding is Not the New Literacy (Chris Grainger) — We build mental models of everything – from how to tie our shoes to the way macro-economic systems work. With these, we make decisions, predictions, and understand our experiences. If we want computers to be able to compute for us, then we have to accurately extract these models from our heads and record them. Writing Python isn’t the fundamental skill we need to teach people. Modeling systems is. Amen!
  3. Let’s Stop Laughing at Groupon (Fortune) — it is much easier to survive a valuation decline as a public company than as a private one.
  4. nsq — Bitly’s open sourced realtime distributed messaging platform.
Comment
Four short links: 27 November 2014

Four short links: 27 November 2014

Scalable Infrastructure, Lens Tech, Javascript Frameworks, and Morality Valley

  1. Stumbleupon’s Big Data Architecture Using Open Source Software (PDF) — not just the list of tools but the functions they implement. Useful!
  2. Innovega — making a contact lens with a tiny bump that acts as a microscope for content shown in glasses. That description, and this link via MIT Technology Review)
  3. How to Pick a Front-End Framework — not unreasonable opinions, largely useful.
  4. [Silicon Valley] Bedevilled by Moral Issues (NYT, registerwall) — given that Silicon Valley tends to copy and paste the mantra, “we’re making the world a better place,” it seem reasonable to expect that tech companies would hold themselves to a higher ethical standard.
Comment
Four short links: 17 November 2014

Four short links: 17 November 2014

Tut Tut ISPs, Distributing Old Datastores, Secure Containers, and Design Workflow

  1. ISPs Remove Their Customers’ Email Encryption (EFF) — ISPs have apparently realised that man-in-the-middle is their business model.
  2. Dynomite (Netflix) — a sharding and replication layer. Dynomite can make existing non-distributed datastores, such as Redis or Memcached, into a fully distributed & multi-datacenter replicating datastore.
  3. After Dockersmaller, easier to manage, more secure containers via unikernels and immutable infrastructure.
  4. Pixelapse — something between Dropbox and Github for the design workflow and artifacts.
Comment: 1
Four short links: 3 June 2014

Four short links: 3 June 2014

Machine Learning Mistakes, Recommendation Bandits, Droplet Robots, and Plain English

  1. Machine Learning Done Wrong[M]ost practitioners pick the modeling algorithm they are most familiar with rather than pick the one which best suits the data. In this post, I would like to share some common mistakes (the don’t-s).
  2. Bandits for RecommendationsA common problem for internet-based companies is: which piece of content should we display? Google has this problem (which ad to show), Facebook has this problem (which friend’s post to show), and RichRelevance has this problem (which product recommendation to show). Many of the promising solutions come from the study of the multi-armed bandit problem.
  3. Dropletsthe Droplet is almost spherical, can self-right after being poured out of a bucket, and has the hardware capabilities to organize into complex shapes with its neighbors due to accurate range and bearing. Droplets are available open-source and use cheap vibration motors and a 3D printed shell. (via Robohub)
  4. Apple’s App Store Approval Guidelines — some of the plainest English I’ve seen, especially the Introduction. I can only aspire to that clarity. If your App looks like it was cobbled together in a few days, or you’re trying to get your first practice App into the store to impress your friends, please brace yourself for rejection. We have lots of serious developers who don’t want their quality Apps to be surrounded by amateur hour.
Comment
Four short links: 30 May 2014

Four short links: 30 May 2014

Video Transparency, Software Traffic, Distributed Database, and Open Source Sustainability

  1. Video Quality Report — transparency is a great way to indirectly exert leverage.
  2. Control Your Traffic Flows with Software — using BGP to balance traffic. Will be interesting to see how the more extreme traffic managers deploy SDN in the data center.
  3. Cockroacha distributed key/value datastore which supports ACID transactional semantics and versioned values as first-class features. The primary design goal is global consistency and survivability, hence the name. Cockroach aims to tolerate disk, machine, rack, and even datacenter failures with minimal latency disruption and no manual intervention. Cockroach nodes are symmetric; a design goal is one binary with minimal configuration and no required auxiliary services.
  4. Linux Foundation Providing for Core Infrastructure Projects — press release, but interested in how they’re tackling sustainability—they’re taking on identifying worthies (glad I’m not the one who says “you’re not worthy” to a project) and being the non-profit conduit for the dosh. Interesting: implies they think the reason companies weren’t supporting necessary open source projects was some combination of being unsure who to support (projects you use, surely?) and how to get them money (ask?). (Sustainability of open source projects is a pet interest of mine)
Comment