"Interaction Design and Connected Devices" entries
The Intimacy of Things
At what layer do we build privacy into the fabric of devices?
Sign-up to attend Solid 2015 to explore the convergence of privacy, security, and the Internet of Things.
In 2011, Kashmir Hill, Gizmodo and others alerted us to a privacy gaffe made by Fitbit, a company that makes small devices to help people keep track of their fitness activities. It turns out that Fitbit broadcast the sexual activity of quite a few of their users. Realizing this might not sit well with those users, Fitbit took swift action to remove the search hits, the data, and the identities of those affected. Fitbit, like many other companies, believed that all the data they gathered should be public by default. Oops.
Does anyone think this is the last time such a thing will happen?
Fitness data qualifies as “personal,” but sexual data is clearly in the realm of the “intimate.” It might seem like semantics, but the difference is likely to be felt by people in varying degrees. The theory of contextual integrity says that we feel violations of our privacy when informational contexts are unexpectedly or undesirably crossed. Publicizing my latest workout: good. Publicizing when I’m in flagrante delicto: bad. This episode neatly exemplifies how devices are entering spaces where they’ve not tread before, physically and informationally. Read more…
A bigger and different way of looking at the IoT
Tim O’Reilly’s Solid Conference keynote highlights the capabilities that will let us shape the physical world.
O’Reilly’s keynote address at the Solid Conference in 2014 explored the human-IoT link. The talk expanded the scope of the IoT, making it clear this isn’t just about individual devices and software — we’re creating “networks of intelligence” that will shape how people work and live.
The talk has become an essential resource for us as we’ve investigated the blurring of the physical and virtual worlds. That’s why we decided to put together a text-friendly version of the presentation that’s easy to scan and reference. And since we think it’s so useful, we’ve made the text version publicly available.
You can download your free copy of “Software Above the Level of a Single Device: The Implications” here. Read more…
There is room for global thinking in IoT data privacy matters
The best of European and American data privacy initiatives can come together for the betterment of all.
Editor’s note: This is part of a series of posts exploring privacy and security issues in the Internet of Things. The series will culminate in a free webcast by the series author Dr. Gilad Rosner: Privacy and Security Issues in the Internet of Things will happen on February 11, 2015 — reserve your spot today.
As devices become more intelligent and networked, the makers and vendors of those devices gain access to greater amounts of personal data. In the extreme case of the washing machine, the kind of data — who uses cold versus warm water — is of little importance. But when the device collects biophysical information, location data, movement patterns, and other sensitive information, data collectors have both greater risk and responsibility in safeguarding it. The advantages of every company becoming a software company — enhanced customer analytics, streamlined processes, improved view of resources and impact — will be accompanied by new privacy challenges.
A key question emerges from the increasing intelligence of and monitoring by devices: will the commercial practices that evolved in the web be transferred to the Internet of Things? The amount of control users have over data about them is limited. The ubiquitous end-user license agreement tells people what will and won’t happen to their data, but there is little choice. In most situations, you can either consent to have your data used or you can take a hike. We do not get to pick and choose how our data is used, except in some blunt cases where you can opt out of certain activities (which is often a condition forced by regulators). If you don’t like how your data will be used, you can simply elect not to use the service. But what of the emerging world of ubiquitous sensors and physical devices? Will such a take-it-or-leave it attitude prevail? Read more…
Who should and should not be talking to your fridge?
A reflection on the social impacts of smarter hardware in the physical world.
Attend Solid 2015 to explore the IoT’s impact on privacy and security.
Here’s the scenario today: I am out of milk, and my refrigerator sits there, mute and unsympathetic. Some time in the 90s, I was promised a fridge that would call the store when I was out of milk, and it would then be delivered while I, ignorant of my dearth of dairy, went about my business. Apparently such predictions were off. Someone forgot to tell my fridge manufacturer to put sensors, software, and networking gear into their products.
But there is hope. The dumb objects in the analog physical world are being slowly upgraded. From the very sexy telemetry systems in new BMWs to the very unsexy pallets of lettuce in a warehouse, Things That Heretofore Were Blind and Mute are getting eyes, ears, mouths, and in some cases, brains. This is evolution, not revolution, and while it is still slow-moving, it’s beneficial to reflect on some of the social impacts of smarter hardware in the physical world. Read more…
The Internet of Things has four big data problems
The IoT and big data are two sides of the same coin; building one without considering the other is a recipe for doom.
The Internet of Things (IoT) has a data problem. Well, four data problems. Walking the halls of CES in Las Vegas last week, it’s abundantly clear that the IoT is hot. Everyone is claiming to be the world’s smartest something. But that sprawl of devices, lacking context, with fragmented user groups, is a huge challenge for the burgeoning industry.
What the IoT needs is data. Big data and the IoT are two sides of the same coin. The IoT collects data from myriad sensors; that data is classified, organized, and used to make automated decisions; and the IoT, in turn, acts on it. It’s precisely this ever-accelerating feedback loop that makes the coin as a whole so compelling.
Nowhere are the IoT’s data problems more obvious than with that darling of the connected tomorrow known as the wearable. Read more…
We need an Internet that performs flawlessly, every second of every day
As we increasingly depend on connected devices, primary concerns will narrow to safety, reliability, and survivability.
Editor’s note: this interview with GE’s Bill Ruh is an excerpt from our recent report, When Hardware Meets Software, by Mike Barlow. The report looks into the new hardware movement, telling its story through the people who are building it. For more stories on the evolving relationship between software and hardware, download the free report.
More than one observer has noted that while it’s relatively easy for consumers to communicate directly with their smart devices, it’s still quite difficult for smart devices to communicate directly, or even indirectly, with each other. Bill Ruh, a vice president and corporate officer at GE, drives the company’s efforts to construct an industrial Internet that will enable devices large and small to chat freely amongst themselves, automatically and autonomously. From his perspective, the industrial Internet is a benign platform for helping the world become a quieter, calmer, and less dangerous place.
“In the past, hardware existed without software. You think about the founding of GE and the invention of the light bulb — you turned it on and you turned it off. Zero lines of code. Today, we have street lighting systems with mesh networks and 20 million lines of code,” says Ruh. “Machines used to be completely mechanical. Today, they are part digital. Software is part of the hardware. That opens up huge possibilities.”
A hundred years ago, street lighting was an on-or-off affair. In the future, when a crime is committed at night, a police officer might be able to raise the intensity of the nearby street lights by tapping a smart phone app. This would create near-daylight conditions around a crime scene, and hopefully make it harder for the perpetrators to escape unseen. “Our machines are becoming much more intelligent. With software embedded in them, they’re becoming brilliant,” says Ruh. Read more…
What you need to know for the hardware-software convergence
Core competencies and essential reading from hardware, software, manufacturing, and the IoT.
As I noted in “Physical and virtual are blurring together,” we now have hardware that acts like software, and software that’s capable of dealing with the complex subtleties of the physical world. So, what must the innovator, the creator, the executive, the researcher, and the artist do to embrace this convergence of hardware and software?
At its core, this is about a shift from discipline toward intent. Individuals and institutions — whether they’re huge enterprises, small start-ups, or nonprofits — must be competent in several disciplines that increasingly overlap, and should be prepared to solve problems by working fluidly across disciplines.
To use Joi Ito’s example, someone who wants to develop a synthetic eye might begin to approach the problem with biology, or electronics, or software, or (most likely) all three together. Many problems can be solved somewhere in a large multidimensional envelope that trades off design, mechanics, electronics, software, biology, and business models. Experts might still do the best work in each discipline, but everyone needs to know enough about all of them to know where to position a project between them.
Below you’ll find the core competencies in the intersection between software and the physical world, and our favorite books and resources for each one.
Electronics for physical-digital applications
- Practical Electronics, by John M. Hughes: To know what’s possible and where to start, it’s essential to understand both the analog and digital sides of electronics. This is O’Reilly’s authoritative introduction to both analog and digital electronics, with information on circuit design, common parts and techniques, and microcontrollers.
- Raspberry Pi Cookbook, by Simon Monk: The Raspberry Pi is rapidly becoming the standard embedded computing platform for prototyping and experimentation, with enough computing power to run familiar interpreted programming languages and widely supported operating systems.
- Arduino Cookbook, by Michael Margolis: The Arduino microcontroller offers a fluid interface between digital and physical; it’s highly extensible and accessible to people with no prior experience in either electronics or code.
An ecosystem of connected devices
Our biggest opportunities as designers and product creators lie in a context-driven approach to designing user experiences.
Editor’s note: This is an excerpt from our recent book Designing Multi-Device Experiences, by Michal Levin. This excerpt is included in our curated collection of chapters from the O’Reilly Design library. Download a free copy of the Experience Design ebook here.
Download a free copy of the Experience Design ebook here.
Consider the hours we spend with devices every day — interacting with our smartphones, working on our laptops, engaging with our tablets, watching shows on television, playing with our video game consoles, and tracking steps on our fitness wristbands. For many of us, the following are true:
- We spend more time interacting with devices than with people.
- We often interact with more than one device at a time.
The number of connected devices has officially exceeded the seven-billion mark, outnumbering people (and toothbrushes) on the planet. By 2020, this number is expected to pass 24 billion. This inconceivable quantity not only attests to the growing role of these devices in our digital lives, but also signals an increasing number of devices per person. Many individuals now own multiple connected devices — PCs, smartphones, tablets, TVs, and more — and they are already using them together, switching between them, in order to accomplish their goals. Ninety percent of consumers use multiple devices to complete a task over time (PDF). For example, shopping for an item might entail (1) searching and exploring options at home on the PC, (2) checking product information and comparing prices in-store using your smartphone, and (3) writing product reviews on a tablet. Eighty-six percent of consumers use their smartphones while engaging with other devices and during other media consumption activities. Read more…
Hardware is an elusive constraint on user experience
Andrew “bunnie” Huang on understanding the interplay between software, hardware, and the existing supply chain.
Editor’s note: this interview with Andrew “bunnie” Huang is an excerpt from our recent report, When Hardware Meets Software, by Mike Barlow. The report looks into the new hardware movement, telling its story through the people who are building it. For more stories on the evolving relationship between software and hardware, download the free report.
Andrew “bunnie” Huang has a Ph.D. in electrical engineering from MIT, but he is most famous for reverse engineering the Xbox, establishing his reputation as one of the world’s greatest hardware hackers. He sees an evolving relationship between hardware and software.“It used to be that products were limited solely by the capability of their hardware. Early radios, for example, had mechanical buttons that acted directly on the physics of the receiver,” says Huang. “As hardware becomes more capable, the user experience of the hardware is more dictated by the software that runs on it. Now that hardware is ridiculously capable — you basically have supercomputers in your pockets that cost next to nothing — pretty much the entire user experience of the product is dictated by the software. The hardware simply serves as an elusive constraint on the user experience.”
Hardware is “a cage,” says Huang, and good software developers learn to work within the constraints of the hardware. “When I work with programmers on new products, I take the first prototype, put it on the desk and I say, ‘Welcome to your new cage.’ That’s the reality. There’s a hard wall. But we try to build the cage big enough so there are options for programmers. A quad core Android phone with a gigabyte of memory is a pretty big cage. Sometimes when programmers feel constrained, they’re just being lazy. There’s always more than one way to skin a cat in the software world.” Read more…
Physical and virtual are blurring together
Key signals from hardware, software, manufacturing, and the Internet of Things.
This essay updates a November 2013 article. We’ve expanded it in light of the success of our first Solid conference in May 2014, where we tested many of these ideas, and the announcement of our next Solid conference in June 2015. In addition to this update, you can stay in the loop on the latest developments in the space through our weekly newsletter.
Real and virtual are crashing together. On one side is hardware that acts like software: IP-addressable, programmable with high-level procedural languages and APIs, able to be stitched into loosely coupled systems — the mashups of a new era. On the other is software that’s newly capable of dealing with the complex subtleties of the physical world — ingesting huge amounts of data, learning from it, and making decisions in real time.
The result is an entirely new medium that’s just beginning to emerge. We can see it in Ars Electronica Futurelab’s Spaxels, which are LED-equipped quadcopters that make up a drone swarm to render a three-dimensional pixel field; in Baxter, which layers emotive software onto an industrial robot so that anyone can operate it safely and efficiently; in OpenXC, which gives even hobbyist-level programmers access to the software in their cars; and in SmartThings, which ties web services to light switches.
The new medium is something broader than terms like “Internet of Things,” “Industrial Internet,” or “connected devices” suggest. It’s an entirely new discipline that’s being built by software developers, roboticists, manufacturers, hardware engineers, artists, and designers. Read more…