Designing a future of immersive, tangible interaction

A look into a future in which physical and digital converge.

Editor’s note: This is an excerpt by Stephen P. Anderson from our recent book Designing for Emerging Technologies, a collection of works by several authors, curated and edited by Jon Follett. This excerpt is included in our curated collection of chapters from the O’Reilly Design library. Download a free copy of the Designing for the Internet of Things ebook here.

In the opening scenes of the Superman movie Man of Steel, one of the many pieces of Kryptonian technology we see are communication devices whose form and shape is constantly reshaping — a tangible, monochromatic hologram, if you will. Imagine thousands of tiny metal beads moving and reshaping as needed. Even though this makes for a nice bit of sci-fi eye candy, it’s also technology that MIT’s Tangible Media Group, led by professor Hiroshi Ishii, is currently exploring. In their own words, this work “explores the ‘Tangible Bits’ vision to seamlessly couple the dual world of bits and atoms by giving physical form to digital information.” They are creating objects (the “tangible bits”) that can change shape.

Even though the team’s vision of “radical atoms” is still in the realm of the hypothetical, the steps they are taking to get there are no less inspiring. Their latest example of tangible bits is a table that can render 3D content physically, so users can interact with digital information in a tangible way. In one of their video demonstrations, a remote participant in a video conference moves his hands, and in doing so reshapes the surface of a table, rolling a ball around. The technology is at once both awe-inspiring and crude; the wooden pegs moving up and down to define form aren’t that unlike the pin art toys we see marketed to children. Having said that, it’s easy to imagine something like this improving in fidelity over time, in the same way that the early days of monochromatic 8-bit pixels gave way to retina displays and photorealistic images.

I mention this example because it’s easy to diminish the value of tangible interactions when compared to the mutability of pixels behind glass; a single device such as a smartphone or tablet can become so many things, if only at the cost of tangibility. Our current thinking says, “Why create more ‘stuff’ that only serves a single purpose?” And this makes sense. I recall the first app for musicians that I downloaded to my iPhone — a simple metronome. For a few dollars, I was able to download the virtual equivalent of an otherwise very expensive piece of hardware. It dawned on me: if indeed the internal electronics are comparable to those contained in the hardware, there will be a lot of companies threatened by this disruption. This ability to download for free an app that as an object would have cost much more (not to mention add clutter) is a great shift for society.

But…

What if physical objects could reshape themselves in the same way that pixels do? What if one device, or really a blob of beads, could reshape into a nearly infinitesimal number of things? What if the distinctions between bits and atoms become nearly indistinguishable? Can we have physical interactions that can also dynamically change form to be 1,000 different things? Or, at a minimum, can the interface do more than resemble buttons? Perhaps it could shape itself into the buttons and switches of the last century and then flatten out again into some new form. How does the role of interaction designer change when you’re interface is a sculpted, changing thing? So long as we’re looking out into possible futures, this kind of thinking isn’t implausible, and should set some direction.

Nothing new under the sun

While much of this looks to a future in which physical and digital converge, there is one profession that has been exploring this intersection for some time now: museums.

Museums are amazing incubators for what’s next in technology. These learning environments have to engage visitors through visuals, interactions, stories, and other means, which often leads to (at least in the modern museum) spaces that are both tangible and take advantage of digital interactions. The self-directed pace that visitors move through an exhibit pressures all museum designers to create experiences that are both informative and entertaining. And, many artists and technologists are eager to, within the stated goals of an exhibit, try new things.

Take for example the Te Papa Tongarewa museum, in Wellington, New Zealand. Because New Zealand is an island formed from the collision of two tectonic plates, you can expect volcanoes, earthquakes, and all things geothermal to get some attention. As visitors move about the space, they are invited to learn about various topics in some amazing and inventive ways. When it comes to discussions of mass and density, there are three bowling ball–sized rocks ready for you to lift; they are all the same in size, but the weight varies greatly. When learning about tectonic shifts, you turn a crank that then displaces two halves of a map (along with sound effects), effectively demonstrating what has happened to New Zealand over thousands of years, and what is likely to happen in the future. Visitors are encouraged to step into a house in which they can experience the simulation of an earthquake. The common denominator between these and dozens more examples is that through a combination of technology and tangible interactions, visitors are encouraged to interact with and construct their own knowledge.

Imagine the future

Novelist William Gibson once commented that future predictions are often guilty of selectively amplifying the observed present. Steam power. Robots. Many of us are being handed a future preoccupied with touch screens and projections. In “A Brief Rant on the Future of Interaction Design,” designer and inventor Bret Victor offers a brilliant critique of this “future behind glass,” and reminds us that there are many more forms of interaction of which we have yet to take advantage. As he says, “Why aim for anything less than a dynamic medium that we can see, feel, and manipulate?”

To limit our best imaginings of the future, and the future of learning, to touching a flat surface ignores 1) a body of research into tangible computing, 2) signs of things to come, and 3) centuries of accumulated knowledge about how we — as human creatures — learn best. Whether it’s the formal learning of schools or the informal learning required of an information age, we need to actively think about how to best make sense of our world. And all that we know (and are learning) about our bodies and how we come to “know” as human beings cries out for more immersive, tangible forms of interaction. I look forward to a union of sorts, when bits versus atoms will cease to be a meaningful distinction. I look to a future when objects become endowed with digital properties, and digital objects get out from behind the screen. The future is in our grasp.

tags: , ,