"infrastructure" entries

Four short links: 15 May 2015

Four short links: 15 May 2015

Army Cloud, Google Curriculum, Immutable Infrastructure, and Task Queues

  1. Army Cloud Computing Strategy (PDF) — aka: “what we hope to do without having done, to use what we’re doing to them.”
  2. Guide to Technical Development (Google) — This guide is a suggested path for university students to develop their technical skills academically and non-academically through self-paced, hands-on learning.
  3. Immutable Infrastructure is the Future (Michael DeHaan) — The future of configuration management systems is in deploying cloud infrastructure that will later run immutable systems via an API level.
  4. machineryan asynchronous task queue/job queue based on distributed message passing.
Comment
Four short links: 12 May 2015

Four short links: 12 May 2015

Data Center Numbers, Utility Computing, NSA Art, and RIP CAP

  1. We Used to Build Steel Mills Near Cheap Sources of Power, but Now That’s Where We Build DatacentersHennessy & Patterson estimate that of the $90M cost of an example datacenter (just the facilities – not the servers), 82% is associated with power and cooling. The servers in the datacenter are estimated to only cost $70M. It’s not fair to compare those numbers directly since servers need to get replaced more often than datacenters; once you take into account the cost over the entire lifetime of the datacenter, the amortized cost of power and cooling comes out to be 33% of the total cost, when servers have a three-year lifetime and infrastructure has a 10-15 year lifetime. Going back to the Barroso and Holzle book, processors are responsible for about a third of the compute-related power draw in a datacenter (including networking), which means that just powering processors and their associated cooling and power distribution is about 11% of the total cost of operating a datacenter. By comparison, the cost of all networking equipment is 8%, and the cost of the employees that run the datacenter is 2%.
  2. Microsoft Invests in 3 Undersea Cable Projects — utility computing is an odd concept, given how quickly hardware cycles refresh. In the past, you could ask whether investors wanted to be in a high-growth, high-risk technology business or a stable blue-chip utility.
  3. Secret Power — Simon Denny’s NSA-logo-and-Snowden-inspired art makes me wish I could get to Venice. See also The Guardian piece on him.
  4. Please Stop Calling Databases CP or AP (Martin Kleppman) — The fact that we haven’t been able to classify even one datastore as unambiguously “AP” or “CP” should be telling us something: those are simply not the right labels to describe systems. I believe that we should stop putting datastores into the “AP” or “CP” buckets. So readable!
Comment
Four short links: 11 May 2015

Four short links: 11 May 2015

Age of Infrastructure, Facial Expressions, Proof Assistants, and Programmer Talent

  1. Welcome to the Age of Infrastructure (Annalee Newitz) — The Internet isn’t that thing in there, inside your little glowing box. It’s in your washing machine, kitchen appliances, pet feeder, your internal organs, your car, your streets, the very walls of your house. You use your wearable to interface with the world out there.
  2. Facial Performance Sensing Head-Mounted Display (YouTube) — glorious use of an Oculus headset, to capture (for reproduction on an avatar) fine-grained facial expressions. From SIGGRAPH 2015.
  3. Mathematical Proof Assistants — human augmentation in mathematics.
  4. The Programmer Talent Myth (LWN) — Jacob Kaplan-Moss on the distribution of programmer talent and the damage that the bimodal myth causes.
Comments: 3
Four short links: 27 January 2015

Four short links: 27 January 2015

Autonomous Corporations, Abstract Thought, Down Rounds, and Distributed Messaging

  1. Decentralised Autonomous Corporations — Charlie Stross’s near-future fiction of Accelerando comes closer to reality: Malice – revenge for waking him up – sharpens Manfred’s voice. “The president of agalmic.holdings.root.184.97.AB5 is agalmic.holdings.root.184.97.201. The secretary is agalmic.holdings.root.184.D5, and the chair is agalmic.holdings.root.184.E8.FF. All the shares are owned by those companies in equal measure, and I can tell you that their regulations are written in Python. Have a nice day, now!” He thumps the bedside phone control and sits up, yawning, then pushes the do-not-disturb button before it can interrupt again. After a moment he stands up and stretches, then heads to the bathroom to brush his teeth, comb his hair, and figure out where the lawsuit originated and how a human being managed to get far enough through his web of robot companies to bug him.
  2. Coding is Not the New Literacy (Chris Grainger) — We build mental models of everything – from how to tie our shoes to the way macro-economic systems work. With these, we make decisions, predictions, and understand our experiences. If we want computers to be able to compute for us, then we have to accurately extract these models from our heads and record them. Writing Python isn’t the fundamental skill we need to teach people. Modeling systems is. Amen!
  3. Let’s Stop Laughing at Groupon (Fortune) — it is much easier to survive a valuation decline as a public company than as a private one.
  4. nsq — Bitly’s open sourced realtime distributed messaging platform.
Comment
Four short links: 27 November 2014

Four short links: 27 November 2014

Scalable Infrastructure, Lens Tech, Javascript Frameworks, and Morality Valley

  1. Stumbleupon’s Big Data Architecture Using Open Source Software (PDF) — not just the list of tools but the functions they implement. Useful!
  2. Innovega — making a contact lens with a tiny bump that acts as a microscope for content shown in glasses. That description, and this link via MIT Technology Review)
  3. How to Pick a Front-End Framework — not unreasonable opinions, largely useful.
  4. [Silicon Valley] Bedevilled by Moral Issues (NYT, registerwall) — given that Silicon Valley tends to copy and paste the mantra, “we’re making the world a better place,” it seem reasonable to expect that tech companies would hold themselves to a higher ethical standard.
Comment
Four short links: 17 November 2014

Four short links: 17 November 2014

Tut Tut ISPs, Distributing Old Datastores, Secure Containers, and Design Workflow

  1. ISPs Remove Their Customers’ Email Encryption (EFF) — ISPs have apparently realised that man-in-the-middle is their business model.
  2. Dynomite (Netflix) — a sharding and replication layer. Dynomite can make existing non-distributed datastores, such as Redis or Memcached, into a fully distributed & multi-datacenter replicating datastore.
  3. After Dockersmaller, easier to manage, more secure containers via unikernels and immutable infrastructure.
  4. Pixelapse — something between Dropbox and Github for the design workflow and artifacts.
Comment: 1
Four short links: 3 June 2014

Four short links: 3 June 2014

Machine Learning Mistakes, Recommendation Bandits, Droplet Robots, and Plain English

  1. Machine Learning Done Wrong[M]ost practitioners pick the modeling algorithm they are most familiar with rather than pick the one which best suits the data. In this post, I would like to share some common mistakes (the don’t-s).
  2. Bandits for RecommendationsA common problem for internet-based companies is: which piece of content should we display? Google has this problem (which ad to show), Facebook has this problem (which friend’s post to show), and RichRelevance has this problem (which product recommendation to show). Many of the promising solutions come from the study of the multi-armed bandit problem.
  3. Dropletsthe Droplet is almost spherical, can self-right after being poured out of a bucket, and has the hardware capabilities to organize into complex shapes with its neighbors due to accurate range and bearing. Droplets are available open-source and use cheap vibration motors and a 3D printed shell. (via Robohub)
  4. Apple’s App Store Approval Guidelines — some of the plainest English I’ve seen, especially the Introduction. I can only aspire to that clarity. If your App looks like it was cobbled together in a few days, or you’re trying to get your first practice App into the store to impress your friends, please brace yourself for rejection. We have lots of serious developers who don’t want their quality Apps to be surrounded by amateur hour.
Comment
Four short links: 30 May 2014

Four short links: 30 May 2014

Video Transparency, Software Traffic, Distributed Database, and Open Source Sustainability

  1. Video Quality Report — transparency is a great way to indirectly exert leverage.
  2. Control Your Traffic Flows with Software — using BGP to balance traffic. Will be interesting to see how the more extreme traffic managers deploy SDN in the data center.
  3. Cockroacha distributed key/value datastore which supports ACID transactional semantics and versioned values as first-class features. The primary design goal is global consistency and survivability, hence the name. Cockroach aims to tolerate disk, machine, rack, and even datacenter failures with minimal latency disruption and no manual intervention. Cockroach nodes are symmetric; a design goal is one binary with minimal configuration and no required auxiliary services.
  4. Linux Foundation Providing for Core Infrastructure Projects — press release, but interested in how they’re tackling sustainability—they’re taking on identifying worthies (glad I’m not the one who says “you’re not worthy” to a project) and being the non-profit conduit for the dosh. Interesting: implies they think the reason companies weren’t supporting necessary open source projects was some combination of being unsure who to support (projects you use, surely?) and how to get them money (ask?). (Sustainability of open source projects is a pet interest of mine)
Comment
Four short links: 19 November 2013

Four short links: 19 November 2013

Ad Triumphalism, Education Not Transformed, Bookstore Infrastructure, and Tossable Camera

  1. Why The Banner Ad is Heroic — enough to make Dave Eggers cry. Advertising triumphalism rampant.
  2. Udacity/Thrun ProfileA student taking college algebra in person was 52% more likely to pass than one taking a Udacity class, making the $150 price tag–roughly one-third the normal in-state tuition–seem like something less than a bargain. In which Udacity pivots to hiring-sponsored workforce training and the new educational revolution looks remarkably like sponsored content.
  3. Amazon is Building Substations (GigaOm) — the company even has firmware engineers whose job it is to rewrite the archaic code that normally runs on the switchgear designed to control the flow of power to electricity infrastructure. Pretty sure that wasn’t a line item in the pitch deck for “the first Internet bookstore”.
  4. Panoramic Images — throw the camera in the air, get a 360×360 image from 36 2-megapixel lenses. Not sure that throwing was previously a recognised UI gesture.
Comments: 3

The Rise of Infrastructure as Data

Simplifying IT automation

IT infrastructure should be simpler to automate. A new method of describing IT configurations and policy as data formats can help us get there. To understand this conclusion, it helps to understand how the existing tool chains of automation software came to be.

In the beginnings of IT infrastructure, administrators seeking to avoid redundant typing wrote scripts to help them manage their growing computer hordes. The development of these in­house automation systems were not without cost; each organization built its own redundant tools. As scripting gurus left an organization, these scripts were often very difficult to maintain by new employees.

As we all know by the huge number of books written on the topic, software development sometimes has a large amount of time investment required to do it right. Systems management software is especially complex, due to all the possible variables and corner cases to be managed. These in­house scripting systems often grew to be fragile.

Read more…

Comments: 3