Thinking with things

Something is lost when we limit interactions to pressing or clicking — our bodies are capable of so much more.


Editor’s note: we’re running a series of five excerpts from our forthcoming book Designing for Emerging Technologies, a compilation of works by industry experts in areas of user experience design related to genomics, robotics, the Internet of Things, and the Industrial Internet of Things.

In this excerpt, author Stephen Anderson addresses the importance of embodied learning and stresses that those concepts need to extend to the way we design and interact with our increasingly connected environment, noting that the digital devices today are painfully unaware of our many human capabilities.


Stephen Anderson, author, consultant, and creator of the Mental Notes card deck.

You may wonder, “why should we care about tangible computing?” Isn’t interacting with our fingers or through devices such as a mouse or touchscreens sufficient? In a world constrained by costs and resources, isn’t it preferable to ship interactive software, that can be easily replicated and doesn’t take up physical space? If you look at how media has shifted from vinyl records to cassette tapes to compact discs and finally digital files, isn’t this the direction that everything is headed?

Where learning and understanding is required, I’d argue no. And a definite no wherever young children are involved. Jean Piaget established four stages of learning (sensorimotor, preoperational, concrete operations, and formal operations), and argued that children “learn best from concrete activities.” This work was preceded by John Dewey, who emphasized first-hand learning experiences. Other child psychologists, such as Jerome Bruner and Zoltán Dienes, have built on these “constructivist” ideas, creating materials used to facilitate learning. In a review of studies on the use of manipulatives in the classroom, Marilyn Suydam and Jon Higgins concluded in their 1976 report “Review and Synthesis of Studies of Activity-Based Approaches to Mathematics Teaching” that “studies at every grade level support the importance and use of manipulative materials.”

And what of adults, who’ve had a chance to internalize most of these concepts? Using Piaget’s own model, some might argue that the body is great for lower level cognitive problems, but not for more abstract or complex topics. This topic is one of some debate, with conversations returning to “enactivism” and the role of our bodies in constructing knowledge. The central question: if learning is truly embodied, why or how would that change with age? Various studies continue to surface this mind-body connection. For example, studies have found that saying words like “lick, pick, and kick” activates the corresponding brain regions associated with the mouth, hand, and foot, respectively. I’d add that these thinking tools extend our thinking, the same way objects such as pen and paper, books, or the handheld calculator (abacus or digital variety — you choose!) have allowed us to do things we couldn’t before. Indeed, the more complex the topic, the more necessary it is to use our environment to externalize our thinking.

Moreover, there is indeed a strong and mysterious connection between the brain and the body. We tend to gesture when we’re speaking, even if on a phone where no one else can see us. I personally have observed different thinking patterns when standing versus sitting. In computer and retail environments, people talk about “leaning in” versus “leaning back” activities. In high school, I remember being told to look up, if I was unsure of how to answer a question — apparently looking up had, in some study, been shown to aid in the recall of information! Athletes, dancers, actors — all these professions talk about the yet unexplained connections between mind and body.

As magical as the personal computer and touchscreen devices are, there is something lost when we limit interactions to pressing on glass or clicking a button. Our bodies are capable of so much more. We have the capacity to grasp things; sense pressure (tactile or volumetric); identify textures; move our bodies; orient ourselves in space; sense changes in temperature; smell; listen; affect our own brain waves; control our breathing — so many human capabilities not recognized by most digital devices. In this respect, the most popular ways in which we now interact with technology, namely through the tips of our fingers, will someday seem like crude, one-dimensional things. Fortunately, the technology to sense these kinds of physical interactions already exists, or is being worked on in research labs.

tags: , , ,

Get the O’Reilly Hardware Newsletter

Get weekly insight and knowledge on how to design, prototype, manufacture, and market great connected devices.

  • kathysierra

    What a wonderful POV on this, Stephen, and I could not agree more. Hoping to see you pursue more on what I think is the most significant — and most under appreciated / overlooked — attribute and value of physical, manipulative tech. What we do with our body impacts what we do with our mind, and we’ve been neglecting that for far too long.

  • Ben Judy

    This is a fantastic excerpt — I can’t wait to read the book!

    As Things become more connected and tangible computing becomes more prevalent, our understanding of accessibility and universal user experience design must grow wider and deeper.

    A good case study would be my daughter who has been diagnosed with Sensory Processing Disorder (SPD.) It’s a misunderstood and under-researched disorder, but in brief: her ability to sense things (or “make sense” of her own sensations) is not the same as for other people. Her mind-body connection is challenging and unpredictable. You rightly point out that the connections between the brain and body are largely unexplained. It is possible that — for those with SPD — reading or hearing the words “lick,” “pick,” and “kick” may *not* activate the corresponding brain regions in the same way as they do for non-SPD people. Thus, whole-body interaction design patterns that might be appropriate for most people may not be appropriate (e.g. a good experience, or even usable) for my daughter.

    As she grows up in a world populated by new interactive Things, her own confusing struggles with her mind-body connection can be aided greatly by the work of empathetic and informed designers. Or, her life can be made all the more difficult and frustrating by an increasingly complex and unfriendly technological environment.

    In recent years many designers have developed new skills in designing for the sight and sound impaired. Now, how do we design for the full spectrum of people who may — or may not — be able to interact with Things with their other senses? Greater challenges await us as we design into the future!

    • Stephen P. Anderson

      Thanks for your feedback, Ben! I agree, there is much more to be learned about the brain and mind-body connections. Much of our understanding is limited to “what is common” for most people; that, and much of what we know with more certainty is limited to studying those individuals who’ve suffered damage to portions of their brains (see: ).

      I’m just starting to dig into some of the notable differences/exceptions, such as you mention with your daughter. Two recent observations, to add to your own commentary:

      1) One of my side projects is a tangible “appcessory” for teaching early math concepts. Early feedback I received is that while tangible learning may be good for all, it might be even better suited for kids on the autism spectrum, as manipulatives tend to (generally) be better for these kids…

      2) I just heard (at SxSWEdu) reference made to research into how learners with ADHD use different parts of the brain (than the rest of the world) to make sense of new knowledge and form memories… This is important to know as we talk about interacting with visual representations or moving through physical space or…

      Definitely exciting times…!

  • giftzncakes

    The people of India have mind blowing opportunity to make stunning surprise to their loving mother. Send same day cakes to India
    and pamper your dear mother. Your mother will be highly delighted after getting such a beautiful gift.