Embracing hardware data

Looking at the collision of hardware and software through the eyes of a data scientist.

Raspberry Pi Board. Via Wikimedia Commons

Many aspects of a hardware device can be liberally prototyped. A Raspberry Pi (such as the one seen above) can function as a temporary bridge before ARM circuit boards are put into place.

Editor’s note: This is part one of a two-part series reflecting on the O’Reilly Solid Conference from the perspective of a data scientist. Normally we wouldn’t publish takeaways from an event held nearly two months ago, but these insights were so good we thought they needed to be shared.

In mid-May, I was at Solid, O’Reilly’s new conference on the convergence of hardware and software. I went in as something close to a blank slate on the subject, as someone with (I thought) not very strong opinions about hardware in general.

The talk on the grapevine in my community, data scientists who tend to deal primarily with web data, was that hardware data was the next big challenge, the place that the “alpha geeks” were heading. There are still plenty of big problems left to solve on the web, but I was curious enough to want to go check out Solid to see if I was missing out on the future. I don’t have much experience with hardware — beyond wiring up LEDs as a kid, making bird houses in shop class in high school, and mucking about with an Arduino in college.

I went to Solid out of curiosity over what I would find, but also because I have spent a lot of time talking to Jon Bruner, the co-chair of Solid, and I didn’t understand what he had been talking about. I’ve heard him talk about the “merging of hardware and software,” and I’ve heard him say “exchange between the virtual and actual,” or some variation on that, at least a few dozen times. Were these useful new concepts or nice-sounding but empty phrases?

Then there was the puzzling list of invitees to the conference. None of them seemed to fit together, apart from the fact that they dealt with tangible things. Why are cheap manufacturing, 3D printing, the Internet of Things, Fortune 50 industrial hardware companies, robotics, and consumer hardware branding exercises all in one conference? What’s going on here?

After attending the conference, talking to a lot of folks, and doing some follow-up reading, I’ve come up with two main takeaways that are intelligible from my perspective as a software and data person trying to come to grips with what Solid is and what the trends are that it represents. First, the cost to market of hardware start-ups is reaching parity with software start-ups. And second, the material future of physical stuff is up for grabs over the next few decades. I’ll cover the first in this part and the second in the follow-up article.

Cost to market

The first major theme of Solid was that the cost of bringing a hardware start-up to market is at, or will soon reach, parity with software start-ups.

Take a moment to absorb that. Obviously, there are still great challenges in producing a hardware start-up (just as there are with a software start-up) but the cost to market is dropping, and fast. At Solid, I saw a number of consumer and industry hardware start-ups going all the way to market on low to mid six figures.

Take Drop and Birdi, two consumer-facing Internet of Things devices on display at Solid. Drop is an intelligent kitchen scale and accompanying iPad app that help make cooking recipes by weight a snap. Birdi is a smart air monitor that can discern different kinds of smoke, can detect both carbon monoxide and pollen levels, and provide alerts to your phone when its batteries are low. Both are going to market on a shoestring budget. Birdi, for example, got a $50,000 seed round from the hardware incubator Highway1, raised another $72,000 on Indiegogo, and expects to be shipping this fall.

Finding historical information on the cost of going to market for a hardware start-up is tough, but, from what I gathered at Solid, numbers in the millions or tens of millions to market used to be typical. Now, those numbers are for well-financed hardware companies instead of table stakes.

Reducing the user interface on the devices also means reduced manufacturing complexity.

Why is that? Why has hardware gotten so much cheaper to produce than before? I can’t claim to understand all the reasons, but there were many that came up in conversations at Solid. Here’s what I gathered.

First, a less obvious one. Embedding computing used to mean special dialects of C written for embedded systems, or Verilog for describing complicated integrated circuits. More and more, embedded computing means a real CPU, with substantial amounts of memory. Vanilla subsets of C++ can be used for the most numerically intensive things, but interpreted languages, such as Python, Ruby, and JavaScript represent viable paths to embedded software development. I asked around while at Solid, and everyone I spoke with had coded their hardware logic in a “normal” language.

Perhaps more importantly on the software side, many aspects of a hardware device can be liberally prototyped in software. The software on the hardware can be written and iterated on using typical software development practices; complex logic and interactivity can be iterated in browser-based mockups; CPUs can be emulated before a single circuit board is created; and when a prototype gets built, Raspberry Pis and other embedded CPUs can function as temporary bridges before real ARM circuit boards get put into place.

Computing is also being split across devices. Most consumer- or industrial-facing hardware I saw at Solid consisted of the hardware itself, software on the hardware, and a phone app that provided most of the user experience. That means that all of the advances in the last decade in producing mobile software apps can be directly applied to simplifying the design and improving the user experience of hardware devices.

As a data scientist, these are some of the most exciting changes to the hardware space. I don’t know much about hardware, but I do know a thing or two about deploying software at scale to add intelligence to products. As more and more of the intelligence in our hardware moves into common languages running on commodity virtual machines, opportunities will continue to open up for data scientists.

Reducing the user interface on the devices also means reduced manufacturing complexity. A common statement I heard at Solid was that every feature you added to a piece of hardware doubled the complexity to manufacture. How much simpler then is a piece of hardware when it has only one button and no complicated display? As hardware interfaces are moving onto mobile devices, the benefits are twofold: faster, cheaper iteration on the user experience, and faster, cheaper manufacture of the hardware itself. Yet another example of “software eating the world.”

And, of course, the most talked about reason for reduced hardware cost: physical prototyping has gotten easier. Additive 3D printing is the best known case, but desktop cutting, laser cutting, and selective laser sintering are also greatly reducing the complexity of building prototypes.

Before I went to Solid, I wasn’t aware that, traditionally, the path from CAD model to physical reality had to pass through a number of stages that oftentimes required a number of translations. First, the CAD model had to be recast as a series of negatives to cut out, then the negatives had to be translated into a tool path for the actual CNC machine, then the coordinates had to be transformed again and often manually (or via floppy disk!) entered into the computer-controlled cutting machine. Each step represents a lot of time and complexity in translation.

By contrast, the 3D printing path from CAD model to physical prototype is much more direct. Design, press go, and come back in a few hours. Desktop cutting and milling machines are getting progressively easier to use as well, reducing the time to cut out and machine complex parts.

Hardware is sharing more and more of software’s strengths.

Maker spaces, fab labs, and well-equipped university labs are putting more and better prototyping hardware within reach of inventors as well, further reducing the cost to iterate on an idea. Electronics prototyping also looks like it’s getting easier (for example, LittleBits), though I don’t know how much these kinds of tools are being used.

Finally, money and mentorship itself is getting easier to come by. There are now hardware accelerators (like Highway1) that both supply seed capital and connect start-ups with supply chain management in China. Kickstarter, Indiegogo, and dedicated consumer hardware suppliers like The Blueprint are bringing down the cost of capital and helping shape networks of sophisticated inventors.

This — the falling cost of going to market for hardware start-ups — is the “merging of hardware and software.” It might be more precise to say that hardware, both in development and manufacturing (which has been software-driven for a long while), is becoming more and more about software over time. Thus, hardware is sharing more and more of software’s strengths, including cheapness and rapid turnaround time.

Where does data fit into all of this? The theory is that cheap, ubiquitous devices will mean an even bigger explosion in data waiting to be analyzed and acted upon. It was hard to get a bead though on what the timeline is for that and what unique challenges that will pose. Looking further ahead, Charlie Stross has pointed out that, in the next few decades, prices for embedding computing are likely to fall low enough that even adding fairly sophisticated computers to blocks of concrete won’t raise their prices by much.

One nice thing about web data is that, from a machine learning perspective, e-commerce and other clickstream event data is fairly straightforward. Sure, there are some time-series, but I haven’t had to do any digital signal processing or modeling of physical systems in my time as a data scientist. Most machine learning models I have seen in practice assume a fairly high degree of independence between data points that just isn’t true in the physical world.

Nor have I had to deal with power constraints, and while full-fledged embedded CPUs are now ubiquitous, don’t expect to see a Hadoop cluster on your Raspberry Pi anytime soon. I’m starting to think about data flow architectures and how sensor and other kinds of data can play together. I expect it will be a useful skill a few years hence.

Photo via Wikimedia Commons.

tags: , , , ,