Incorporating Tactile Buttons on a Touchscreen Interface
Printer-friendly version
Physical buttons have an indisputable advantage in that they offer an intuitive method of interaction. As I sat typing that sentence at my keyboard, I found that hitting buttons requires little attention and, for the most part, I didn’t even have to look to see where to position my fingers.
Keyboards not entirely unlike the one I’m using at the moment are a familiar sight at kiosks, with a large range of vandal-proof keyboards from companies like DS Keyboard, InduKey and Sasse specifically manufactured to survive the tough outdoor climates kiosks are often exposed to.
You may also have used touch screen interfaces where, when more complicated (and usually lexical or numerical data input like addresses or voucher codes) have to be entered, a keyboard appears on the screen, which you can use to enter one character at a time. I’ve encountered this scenario when collecting train tickets I pre-ordered online at the station. Despite the visual keyboard, this interface isn’t all that simple to operate, and I usually have to re-enter my authorisation code several times before I finally get it right.
While a touch screen interface has proved itself immensely popular in self-service and digital signage solutions, perhaps its biggest downside is its lack of tactility. Of course I’m not the only person to notice this and there have been various new technologies offered to get round this. 3M Touch Systems have a Microtouch Capacitive Touch Sense System which has a touch interface incorporating tactile feedback technology which, although it doesn't help with targeting accuracy, does provide reassurance that touch has been noticed, measured and responded to.
Wouldn’t it be great if you could incorporate the physicality, and familiarity, of keyboards into a touch screen interface? Unfortunately, most interaction on large screens requires uninterrupted, flat surfaces and buttons are rarely integrated with the surface as they would be inappropriate most of the time and therefore waste valuable space on the surface.
In a recent paper “Providing Dynamically Changeable Physical Buttons on Visual Display” by Chris Harrison and Scott Hudson of Carnegie Mellon’s Human-Computer Interaction Institute, a technique that seeks to occupy the space between physical buttons and touch screen interfaces is explored, one which can offer the flexibility of touch screens, while retaining the beneficial tactile properties of physical interfaces. The primary goal of their tactile display was to provide the flexibility of touch screens with the tactile benefits of physical buttons.
Having identified the absence of inherent tactile qualities in touch interfaces, they investigated a visual display that contains deformable areas, so that it can produce physical buttons when required. Such tactile features would be dynamically brought into and out of the interface, thereby creating a solution with the full impact of a visual display (they used rear projection), as well as allowing for multi-touch input (through an infrared lighting and camera set up behind the display), and the potential for physical input through dynamic, pop-up buttons.
These tactile buttons can operate like real buttons in a keyboard: input takes place by running your fingers over the tactile clues to locate the right buttons rather than using your eyes to locate the correct target.
How did they do this?
Using pneumatic actuation, they manipulated aspects of physical form and appearance so buttons could be brought into and taken out of an interface as needed, being dynamically modified under program control. Alternative methods of tactile actuation include pin matrices, vibration, piezo-electric actuation, ciliated surfaces, electromagnetic actuation with voice coils or a simple solenoid.
What are the advantages of pneumatic actuation?
One of the things that makes the pneumatic actuation technology used in this research more attractive than the above is that the materials required are inexpensive and can be simply constructed. No motors, wires or conduits had to be placed behind the display to achieve physical actuation, which would have ruled out the use of rear projection and increased the construction complexity. Diffused illumination multi-touch sensing in combination with rear projection enables considerable scale at little extra cost or complexity.
So how does it work?
Full adhesive covering on the backing layer produces equivalent negative and positive tactile states.
The simplest design uses an acrylic layer with cut-out areas with a latex layer attached on top with adhesive. When the air chamber is negatively pressurised, the latex deforms inwardly, forming concave button-like features. When a positive pressure is applied, the latex would stretch outwards forming convex features. The shape of the deformation is determined by the openings cut into the acrylic backing layer. When no pressure is applied, the display is flat.
Complex masks can be used to create distinct positive and negative states.
Physical features can be actuated by a latex diaphragm behind the backing layer.
A radically different tactile effect can be created by reversing the latex and backing layers and cutting out features in the backing layers while leaving the pieces in situ. A clear latex sheet would be attached to the underside with adhesive. When a positive pressure is applied below this sheet, the latex deforms outwards, which in turn pushes the acrylic elements suspended on top of the layer outwards as well. This would results in buttons which would even depress when pushed. A material without matching cutout features would be needed on top to act as a projection surface.
Please see the video below for an indepth discussion of how the technology works:
Are there are drawbacks?
Because the tactile features are statically arranged, the graphical interface would have to conform to what the display offered. Furthermore, the reliance on rear projection means that a 42” display would require around 40cm of depth. This, however, may not pose such a problem to large, semi-permanent installations like ATMs and check-in kiosks.
What sensing input is used?
Several sensing techniques are possible with pneumatic tactile displays, such as embedding small physical buttons or capacitive areas beneath the target. In the study they used a camera-based sensing technique, to minimise complexity and ensure that there was no interference with the rear projection.
And what about multi-touch?
Multi-touch is achieved with camera sensing of diffused infrared illumination.
This technique is called Diffused Illumination Multi-touch sensing because the display is constructed from infrared-transmissible materials (latex and acrylic). An alternative technique, Frustrated Total Internal Reflection, would not be possible because the acrylic layer is not contiguous.
What kind of feedback do you get?
When a user depresses a pneumatic tactile feature there is a change in air pressure in the cavity behind the backing layer. Because the amplitude of pressure change is related to button displacement distance, pressure- based interaction can be created. Shallower presses produce smaller pressure increases and vice versa.
This can also be used to create both on-click and on-release finger events as the pressure decreases when the tactile feature is released.
Although their simple protoypes show only one or two tactile states at a time, it is possible to have several unique tactile states in a single spot, using the intra-latex bladders technique.
Where could this technology be put to use?
In kiosks, of course! When text entry is required on a touch screen, a physical keyboard could be raised up from the flat surface to aid in typing and retract when complete.
In ATMs, the classic number keypad and arrows on the sides of the screen could be replaced by an easier-to-use tactile display which would provide a richer user experience, and also help those with impaired vision or motor skills, as tactile cues assist greatly in target location.
- user enters PIN number on a raised number keypad
- after four digits are entered, the enter can cancel button would rise from the surface, indicating that the user can proceed (cancel could be given a dedicated chamber so that users can cancel at any time in the process)
- three buttons pop up if the PIN number is successful, offering the user the possibility to check their balance, withdraw or make a transfer.
- A keypad would rise gain wherever number input was required.
- Peripheral buttons could be used for other functions , like selecting currency type or which account to withdraw from
- A quick cash interface could also be used
Flat screens were found to be the most difficult surface to use because of the absence of tactile clues meaning that much more visual attention was required from the user. In the study, positively pressurised pneumatic buttons performed the best in their preliminary user study, even as well as physical buttons. Physical buttons were actually found to require more glances to confirm location of the buttons than those of the pneumatic buttons.
Paper Citation:
Harrison, C. and Hudson, S. E. 2009. Providing Dynamically Changeable Physical Buttons on a Visual Display. In Proceedings of the 27th Annual SIGCHI Conference on Human Factors in Computing Systems (Boston, Massachusetts, USA, April 4 - 9, 2009). CHI '09. ACM, New York, NY.
Interactive Digital Signage
