Could Augmented Reality be the Future for Digital Signage?

Version imprimable

Dr. Michael Klein, INM-Institute for New Media and aurea hap.dig solutions, discusses some of the opportunities of integrating augmented reality with digital signage. Hear more from Dr. Michael Klein at the Open Forum at DIGITAL SIGNAGE EXPO 2010!
 

Just imagine! – in the near future, digital signage displays will become the place where the digital and the real world will blend, giving rise to a fascinating new user experience of mixed realities. Your customers will be able to hold physical objects, such as catalogues, magazines, brochures or any type of packaging, with digital augmentation in their hands. But this is more than just a pipe dream. With an augmented reality-enhanced display system, any kind of digital animation can be augmented in real time with real-world video images to form a totally unique experience. Catalogues can show three-dimensional computer simulations of the products they contain, stories will spring to life and video films will overlie the images in magazines. Brochures will become like magic books when users interact with computer models of the products on offer. The possibilities are endless – just think of your favourite sci-fi movies and you’re maybe half way there already.
 
This fascinating future is not so far away. The only thing left to be done is to merge two already existing technologies into one holistic system. One part supplies digital signage display technology in the form of highly integrated systems. The other part provides a technology called ‘Augmented Reality’ or ‘Mixed Reality,’
 
For the purpose of this overview, digital signage (or DS) means all forms of digital media, such as advertising and information systems, information kiosks, in-store marketing, wayfaring systems or wall screens in public spaces. Modern DS displays are capable of running streaming video content in full screen formats, and are already based on integrated systems with built in computational and graphical power suitable for hosting even the most resource-demanding software and broadband network features, which are useful for remote monitoring and editing of content by content management systems, and also suitable for broadband real-time content feeds. To enhance DS systems for the needs of context-sensitive animations or interactive user participation, it is necessary to built in interaction devices. One existing option is to upgrade DS platforms with near-field communication devices like Bluetooth. In the case of augmented reality enhancements, high performance webcams or video cameras also need to be added.
 
Off the shelf?
 
In the end, an augmented reality enhanced digital signage system is a full-grown multimedia computer system with a video-cam, state-of-the-art CPU and graphics board and broadband network access. And these highly integrated systems are already available on the market.
 
‘Augmented Reality,’ or AR, is a technology that embeds virtual information of all kinds into the real environment that is represented by a live video stream. In front of a digital screen, from video walls to digital signage displays or mobile devices, the user experiences fascinating real-time augmentations right from his or her individual point of view. Three-dimensional digital and interactive models are convincingly linked into the real environment. This link is mostly a spatial relation in size, position and perspective between the augmentations and the real objects, i.e. printed images or so called markers.
 
If this sounds too technical just know that Gartner’s identifies Augmented Reality as one of the Top Ten Disruptive Technologies for 2008 to 2012 (Emerging Trends & Technologies Roadshow, May 28, 2008, Melbourne, Australia). And Gartner defines a disruptive technology as one that causes major change in ‘the accepted way of doing things,’ including business models, processes, revenue streams, industry dynamics and consumer behaviour.
 
AR applications are context-sensitive environments that entangle real world objects with virtual world computer models to create a holistic mixed-reality experience. For the human-computer interface the real world objects in the hand of the user are presented by live video stream images shown on digital signage displays. For the computer-generated imagery three-dimensional models or animations provide the digital input. In the background, the optical tracking and registration of the real-world reference objects in the video images is either done by markers (which look like two-dimensional bar code pictures) or marker-less objects, which means that the software is trained to recognise any A-priory given image. The presentation of the compound images is done by special rendering software.
 
The challenge is the real-time augmentation of the digital models so that they are presented in the correct size, position and perspective in respect to the moving target. Though there is still a lot of development in the field, AR software is already running effortlessly on the standard multimedia computer systems described above.
 
The ultimate goal of augmented reality research is to integrate graphical objects into real environments such that the observer can no longer distinguish between the real and the virtual.
 
As the technology gets more and more mature, and the costs come down, the convergence between 3D, AR and DS technology will soon be offering very attractive marketing and sales support solutions. To understand the landscape, let’s take a short look at some already existing prototypes on the borderline between R&D and market applications.
 
One of the first AR-enhanced DS applications tested for market use is the ‘Magic Mirror.’ The setting is simple, but the effects are overwhelming. All you need is a DS display and an integrated DS system with a mounted video camera. The tracking of the real world, which in the case of the Magic Mirror is the customer in front of the display, is done by face tracking algorithms. The AR software positions any kind of digital object or animation from a predefined database onto the head of the customer. Advanced applications integrate virtual characters sitting on the customers´ shoulders and talking to them. There is no limit for creativity and any contextually needed digital models or animations can be applied. One potential field of use of the Magic Mirror is for location-based and subject-related marketing and sales measures. The launch of a new book, film or game could be accompanied by Magic Mirror devices in shops, museums or cinemas, for example.
 
Other AR-enhanced DS applications are the so called ‘Living Objects’ kiosk systems, which allow, for example, to show ‘what’s inside the box’ or ‘animate the packaging.’ Consumers can take an unopened packet and while presenting, le’ts say, a construction set to a Living Objects AR kiosk, a computer animation assembling the individual parts of the set to complete the final model will be shown as an augmentation to the real box in the display. Again, all you need is an integrated DS system with a video camera and a built in AR enhancement.
 
One last example is the ‘Living Prints’ application, which animates the content of printed documents through augmented video or 3D computer simulations. AR-enhanced DS systems can be trained to recognise all types of printed images to use them as real-world references for augmented visualisation. Brochures come to life, books show computer animations, catalogues present 3D models and ad-cards give your customers a personalised message in their hands. Living Prints solutions are particularly effective for brand products distributors, retailers, the gaming industries or museums and exhibitions.
 
To wrap up, let’s have a look over the shoulders of the developers in the R&D labs. There are at least two developments in progress of serious relevance to digital signage and kiosk solutions.
 
For one, in the very near future, augmented reality will go mobile. This means that mobile devices, i.e. smart-phones with videocams, will become the platforms for AR applications themselves. And these mobile devices can also be integrated as human-machine interfaces for AR-enhanced DS tools. The advantages for marketing and sales support solutions are obvious.
 
The second development is even more promising for AR-enhanced DS – work on the ability of AR software to upload models and animations on the fly, either from local databases or from remote network servers ( the server-based computing- paradigm). Driven by user interactions, the number of digital scenes to be used for the augmentation thus becomes nearly infinite. When this technology becomes freely available, we will come close to the idea of the true reality-virtuality continuum, or the “v-real” (virtual-real) identities that the theoreticians have long envisioned for our future.
 
Hear more at the Open Digital Signage Content and Solutions Forum at DIGITAL SIGNAGE EXPO 2010!