Multi-modal Navigation through Spatial Information

We show how multi-touch hand gestures in combination with foot gestures can be used to perform typical basic spatial tasks within a Geographic Information System (GIS). The work is motivated by the high complexity of User Interfaces common GIS usually display and which requires a high degree of expertise form its users. Recent developments in interactive surfaces that enable the construction of low cost multi-touch displays and relatively cheap sensor technology to detect foot gestures allows the deep exploration of these input modalities for GIS users, with medium or low expertise. Combining multi-touch hand and foot interaction has a couple of advantages and helps also to rethink the use of the dominant and nondominant hand. In pure multi-touch hand interaction systems the nondominant hand often sets the reference frame that determines the navigation mode, while the dominant hand carries out the precise task. Since in this case one touch is only used to define a certain mode, the advantages of multi-touch are not fully exploited. Foot gestures can be used to provide continues input for a spatial navigation task (such as panning or tilting), which is more difficult to provide with the hands in a natural way. In this paper propose how to combine those with a small set of foot gestures to improve the overall interaction with spatial data. INTRODUCTION & RELATED WORK Multi-touch has great potential for exploring complex content in an easy and natural manner. In general, tangible user interfaces (TUIs) provide physical form to digital information and computation. People have developed sophisticated skills for sensing and manipulating their physical environments (Ullmer and Ishii 2000). However, most of these skills are not employed in interaction with the digital world today. Multi-touch interaction with computationally enhanced surfaces has received considerable attention in the last few years. The rediscovery of the Frustrated Total Internal Reflection (FTIR) principle (Han 2005), which allows for the building of such surfaces at low cost, has pushed forward the development of new multi-touch applications quickly. Some designers of these applications make use of the geospatial domain to highlight the viability of their approaches. This domain provides a rich and interesting testbed for multi-touch applications because the command and control of geographic space (at different scales) as well as the selection, modification and annotation of geospatial data are complicated tasks and have a high potential to benefit from novel interaction paradigms (UNIGIS 1998). These are central tasks in a GIS (as in any interactive system) (Maceachren et al. 2004, Wasinger et al. 2003). One particular problem that conventional Geographic Information Systems have is the low expressiveness and the significant ambiguity of single pointing gestures (provided by the traditional WIMP paradigm (Myers 1988)) on geospatial data. GIS often maintain data organized in different overlapping layers, each containing information of a spatial feature class (e.g. streets layer, regional boundaries layer, national boundaries layer, etc.). A single pointing gesture is often not enough to precisely identify the intended object of a selection. If one considers also the multitude of operational commands (such as topological operations, annotations or geometric modifications) supported by a GIS, it is clear that these systems can only be operated by an expert user, even if temporal information on the pointing gesture is available (Florence et al. 1996). ADVANCES NAVIGATION METHODS USING BOTH HANDS AND FEET Multi-touch gestures provide much more information than a single point gesture, i.e. they allow the user to explore multiple regions of contact and their temporal change with respect to each other to increase the expressiveness of the interaction (Schöning et al. 2008). Combining hand and foot gestures has several advantages. Hand gestures are good for precise input regarding punctual and regional information (Pakkanen 2004). It is however more difficult to input continuous data with one or several hands for a longer time. For example, panning a map on a multi-touch wall is usually performed by a “wiping”-gesture. This can cause problems if the panning is required for larger distances, since the hand moves over the surface and when it reaches the physical border it has to be replaced and then moved again. In contrast foot interaction can be provided continuously by just pushing the body weight over the respective foot. Since the feet are used to navigate in real space such a foot gesture has the additional advantage of being more intuitive in the sense that it borrows from a striking metaphor. Motivated by this finding we developed a method in which users can navigate through spatial data in combination with their hand and with their feet by shifting their weight over their feet on a Wii Balance Board1 (as shown in Figure 1). Tilting is performed just with the feet and two-handed gestures can be used for more appropriate tasks, such as zooming or region. The video2 shows the application in use. Figure 1: User Interacting with Nasa World Wind (Java SDK) using both hand and feet.