Talking to big machines

What “design beyond the screen” means for the industrial Internet.


GE’s 3.2-103 wind turbine analyzes tens of thousands of data points every second and communicates seamlessly with neighboring turbines, service technicians, and operators.

One of the core ideas we set out to explore at Solid is “design beyond the screen” — the idea that, as software moves into physical devices, our modes of interaction with it will change. It’s an easy concept to understand in terms of consumer electronics: the Misfit Shine activity tracker has a processor and memory just like a computer (along with sensors and LEDs), but you don’t control it with a keyboard and monitor; you interact with it by attaching it to your clothing and letting it gather data about your movement. At its most elegant, design beyond the screen minimizes interaction and frees humans to spend their mental energy on things that humans are good at, like creative thinking and interacting with each other.

Design beyond the screen is a much broader and more transformative concept than just that, though: it encompasses changes in the relationships between humans and machines and between machines and other machines. Good design beyond the screen makes interaction more fluid and elevates both people and machines to do their best work. The impact of good design beyond the screen could be huge, and could extend well beyond consumer electronics into heavy industry and infrastructure.

Beth Comstock, General Electric’s CMO, touched on that idea at our Solid conference a few weeks ago. Speaking about GE’s efforts in creating intelligent machines, she said, “we’re granting superpowers to the machines that serve us and the humans that use them.”

At the heart of those efforts is experience design that improves the way machines interact with each other and with humans. Software intelligence can make sense of vast streams of data that modern, instrumented machines generate and present actionable advice to human operators. “We’re building more sophisticated sensors into our brilliant machines all the time, but we need to ensure we can understand what they’re saying,” Comstock told me later. “We have to simplify the output of these machines and give them better contextual awareness.”

In Comstock’s vision, software intelligence improves machines in two respects: it makes them both more selfish and more selfless. “Selfish machines” monitor themselves and ask for help when they need it, as when a jet engine calls for service because vibration sensors have detected early signs of wear. Human operators have typically performed this function, observing, for instance, that a combination of rising temperature and falling pressure suggests an oil leak. Not only can software monitor machines at extremely high accuracy and frequency, it can also present nuanced, abstracted conclusions to human users.

“Selfless machines” coordinate across networks and modify their own operation to improve the output of the entire system. GE’s wind turbines, for instance, can adjust themselves so they don’t block wind flows to the turbines around them. This is also a function that’s usually been performed by central human load balancers. Moved into software, it can be handled frequently and with relentless optimization at any of the many levels of software that touch these machines — from the embedded systems in the machines themselves to the global networks that control them.

So, what of the humans who operate these machines — the thousands of clipboard-bearing jobs that might be made redundant by machines that can report directly to operating software? “The roles of managers, engineers and data scientists will not just change,” Comstock told me, “but will have to align and become even more interdependent than ever. Some jobs and skills will become obsolete, but it’s likely that more will be created.”

In the meantime, machines and humans will both adapt as new forms of human-machine interaction emerge. “You’re going to have to learn how to speak machine,” said Comstock, but machines are evolving quickly; “they speak in a babble of languages right now,” she said, but “we’re on the verge of finding the Rosetta Stone.”

tags: , , ,

Get the O’Reilly Hardware Newsletter

Get weekly insight and knowledge on how to design, prototype, manufacture, and market great connected devices.

  • Jon – an excellent post! Most blogs/articles on IoT are regurgitating the same topics. It is an interesting concept of selfish and selfless machines – and very well explained. My company, Grid2Home, designs and builds software that makes machines in the Smart Grid and Energy “smart”. Your categorization will help us articulate our value better….