Sometimes one screen isn't enough

A look at 10 multi-screen projects and experiments.

This is part of an ongoing series related to Peter Meyers’ project “Breaking the Page: Transforming Books and the Reading Experience.” We’ll be featuring additional material in the weeks ahead. (Note: This post originally appeared on A New Kind of Book. It’s republished with permission.)

I’ve been fiddling with the idea of using multiple displays to give a presentation — putting different slides on different screens. One design sketch — working title: “Documan” — has gotten some chuckles around my office (yes, I work alone):

Illustration of five iPads attached to a man, standing next to a computer display
Man-mounted iPads, plus a nearby monitor. A few possibilities not shown: each iPad could contain images, not just text; objects could move between iPads or from iPad to monitor; and presenter could rotate one or more iPads.

Why on earth does the world need to see a man strap on a half dozen iPads? And, more importantly, what kind of message would benefit from a rig like this?

Beats me. But I do think that content experiments, designed expressly for the screens we all use — rather than our ancestors’ print pages or single PowerPoint slides — are the best way to figure out how stories and teaching change when they move onto the touchscreen.

I’ll spare you, for now, the words and images I’m testing out to fill those screens. (One teaser, though: think about how easy Keynote for iPad makes it to build an action that exits screen right and enters screen left. Now, if you could just get the timing right when using two iPads …).

Clearly, I’m not the only guy playing around here. Ahead, I round up a few content confections that span multiple screens. Some involve separate physical displays, others use different virtual windows. Not all of this stuff is new. But I find it thought provoking how creative types are using the small, medium, and large screens that increasingly coexist near each other.

iPad + projector

Joe Sabia calls himself an “iPad storyteller” — love it! He showed off his stuff at a recent TED talk where he uses his tablet and a variety of different apps (iBooks, a drawing app, Google Earth, Photos, and so on) to entertain an audience that is variously fixed on him, the big projector screen which his iPad is attached to, and the iPad’s display itself.

iPad + magician

Sleight-of-hand artist and iPad maestro Simon Pierro pulls off some awfully clever tricks with his iPad and a real tennis ball, a glass of milk, and a weather forecaster’s hair (she’s on a video inside the iPad). I have no idea what’s magic, what’s video editing trickery, or what he and the iPad are actually doing. And, you know what? It doesn’t matter. What he demonstrates here is how man and machine can team up to entertain in really innovative ways. Don’t miss his part two, where he — sorta/kinda — sheds light on what he’s done.

iPad-powered window displays

Gin Lane Media filled up three of Saks 5th Avenue’s storefront windows with 64 iPads and nine 27-inch displays.

iPad/iPhone partnerships

A few apps use the big and small screen of a tablet and a smartphone in tandem. The iOS app Scrabble, for example, lets you conduct group games in which the iPad serves as publicly viewable board and the iPhone is each player’s private letter stash. Remote Palette is a painting app where the iPad is the canvas and the iPhone is the paint palette.

Multiple browser windows

The band Arcade Fire worked with director Chris Milk to compose this mind-blowing HTML5-powered interactive video for its song “We Used to Wait.” You give this web app the address of the house or building where you grew up in. It then whips together a custom-built video (woven around some stock footage) that incorporates Google Maps footage of your old neighborhood and other graphical magic mashups … all in multiple browser windows of various sizes. (It only works in the Chrome browser.) If you like this one, you’ll love, which uses snapshots of you from your laptop’s webcam, and your Facebook and Twitter feed, to compose a multi-window extravaganza. It all culminates in a mosaic of your face built out of pix pulled from your social media feeds.

Multi-screen patterns

Here’s a pattern-style analysis of different content and interaction designs for multiple displays, from the basic (how Amazon uses Whispersync to keep book location and notes coordinated across a user’s different reading devices) to some innovative software that helps end users take an image, chop it up, and display it on their own collection of displays. That’s what the next item is about.

Junkyard Jumbotron

Free to use (beta) software from some MITers that automatically splits up an image and displays it on whatever collection of screens (smartphones, tablets, PCs) you assemble. This demo shows it in action.

The multi-screen experience

Here’s a five-minute video, with a bunch of TV and consumer electronics execs and analysts. Nothing hugely revelatory, but a nice little brain-tickler about how we are entering an age wherein audience and content producers alike are thinking about how to create and consume stories that play across displays of many different sizes.

Splitscreen: A Love Story

Heartwarming. Winner of a Nokia smartphone video-making contest, this video shows how split-screen stories can add up to more than the sum of their parts.

Google Wave cinema: “Pulp Fiction”

Not really — okay, not at all — safe for work, but a really nifty example of how innovative, multi-pane software (in this case, the soon-to-be late Google Wave), allowed one artist to take a scene from “Pulp Fiction” and render it within this program, weaving in videos, image, text, and maps.

TOC NY 2012 — O’Reilly’s TOC Conference, being held Feb. 13-15, 2012, in New York City, is where the publishing and tech industries converge. Practitioners and executives from both camps will share what they’ve learned and join together to navigate publishing’s ongoing transformation.

Register to attend TOC 2012


tags: , ,