Look through the AR window

“Augmented Reality is your window to the world,” suggest some companies who have released applications for the iPhone 3G S and Android handsets over the past few months. As we have witnessed since the launch of Layar in June, the world needed another “window.” Or, perhaps we needed another browser? Have you heard about the World Surfer by GeoVector?

Rather than focusing entirely on the window itself, it’s time to look at the value the information in the window is bringing to the user.

Let’s study the user’s interaction with digital information about their world on a deeper level. What are people likely to need AR applications to do for them? The hottest applications for AR in the next year will closely resemble familiar human interactions with the physical world.

High Level Taxonomy

In order to really understand the requirements of AR applications, we need clearly defined application categories.

One stab at an AR application taxonomy distinguishes between those applications which require a very highly controlled, perhaps a highly “instrumented” user environment, and those which do not. Intersense is one of the leading providers of technologies for AR applications that require knowing the precise position of user and objects in a reference framework. Examples of highly-instrumented environments are a technology-assisted operating theater in a hospital, a virtual assembly line in a manufacturing plant, an oil drilling platform, a museum with AR exhibits or the space in which visitors can experience an AR-enhanced ride within a theme (amusement) park.

Then there are uncontrolled environments: anywhere that lacks unique identifiers placed in advance for the benefit of the AR user’s application, and where the environmental conditions may rapidly change.

As I was preparing the description of the upcoming Mobile Magic Wand workshop, I discovered that there also needed to be a distinction made between what I call “mobile AR” and that which the chairs of the Let’s Go Out workshop, a workshop conducted in parallel with the Mobile Magic Wand meeting, call simply “outdoor AR.”

Here’s what we came up with: Mobile AR applications or services can use an off-the-shelf hardware platform, such as a mobile phone, UMPC or personal digital assistant with pre-integrated sensors (GPS or camera, for example), or a custom-designed system of any specification as long as the user can carry it without assistance and the device is not connected to its database by a physical cable of fixed length. The access to digital information can be mediated by a wireless or cellular network connection or the digital information necessary for the AR application can all be kept local to the user’s device (pre-loaded).

Mobile AR can be both indoor and outdoor. The scope of “Outdoor AR” includes that portion of mobile AR for which the use case is outside of any building or shelter. There may be some Outdoor AR applications which are not highly portable. Mobile AR includes only that portion of Outdoor AR which is accessible with a device a user can carry without assistance, which is not connected to a server by means of a physical cable and is entirely outside a highly-instrumented environment.

Here’s a photo of me wearing the LifeClipper2 system.

Christine Perey using lifeClipper2 in Basel [photo credit: Jan Torpus]

It’s mobile, at least I could carry it unassisted, and I’m outdoors.

Mobile AR applications

Let’s explore the next level in the Mobile AR application taxonomy. There are professional applications for Mobile AR: this covers all the uses for a mobile AR system by people doing their job.

In one of the LifeClipper2 scenarios, for example, urban planners wear the system to experience (during the design phase of a project) the changes they propose to make in the real environment. These changes could include introducing new vegetation, removal of existing buildings and, perhaps, addition of new facilities. The mixture of simulated objects with the real world is very fluid and, in some instances, even includes changing the acoustic properties in the space.

In the class of Mobile AR for consumers, there are navigation applications or services. Primarily pedestrian navigation, these way-finding applications help us get to the nearest bus stop, public rest rooms, ATM machine, a doctor’s office, coffee shop, McDonald’s and other fixed point which we might need to find in geo-space. We could find these points the old-fashioned way, by asking someone or using a map, but is easier for some to navigate by means of a screen and using the phone’s GPS and compass than on a digital or paper map view.

The next category of Mobile AR consumer applications is associated with having fun: games which involve the user’s natural environment as stimuli or just weave together the natural world and synthetic game objects and players. Think of the Kweekies game, or the AcrossAir virus killer 360 application. In this rich and expanding category, the user’s interaction with the surroundings by way of a consumer mobile telephone or PDA is playful, or at the very least involves earning points in some way. Frequently the metaphor is pointing and shooting or tapping. I’m not implying that these applications aren’t purposeful because having fun is a widely-felt human need, but it is distinctly different from other application categories.

Social AR is the third category which is ripe with possibilities. It is at the intersection of social networks and AR interfaces. Social AR applications will seek to fulfill the human need to find people and to share with friends and fellow inhabitants something personal about ourselves. Social AR will permit us to annotate the places we co-occupy or which we have occupied at different times. Some Social AR applications will arise from location-based social network services such as Yelp’s Monocle feature in its iPhone application.

Mobilizy has made a big contribution in this category with the release of Wikitude.me. Expect there to be many other examples of social AR coming out in the near future. Social AR will undoubtedly also have aspects in common with game applications when two or more people are using their AR interfaces to play with one another.

There are more AR application categories. Can you suggest a few? What are your favorite Mobile AR application segments?