Multi-touch displays: design, applications and performance evaluation

When interacting with a regular desktop computer, indirect devices such as a mouse or keyboard are used to control the computer. Results of the interaction are displayed on a monitor. Current operating systems are restricted to one pointing device. With the introduction of multi-touch, a new form of human computer interaction is introduced. Multi-touch combines display technology with sensors which are capable of tracking multiple points of input. The idea is that this would allow users to interact with the computer in a natural way. Due to recent innovations multi-touch technology has become affordable. For this project a camera based multi-touch device has been constructed at the Scientific Visualization and Virtual Reality group of the Section Computational Science of the Universiteit van Amsterdam. To perform multi-touch point tracking we used Touchlib, a free open source multi-touch framework. To demonstrate the possibilities of multi-touch input technology we created new rich media applications which are controlled by gestures. Existing software can benefit from multi-touch technology, this is demonstrated in a real-time fluid simulation model and the geographical application NASA World Wind. Multi-touch systems are often stand alone systems that do not have external input devices attached. In order to simplify common tasks, a gesture recognition engine has been designed (Gesturelib). Through a set of experiments we evaluate how multi-touch input performs on tasks compared to conventional mouse input. Unlike interaction on a desktop computer multi-touch allows multiple users to interact with the same devices at the same time. We present measurements that show how collaboration on a multi-touch table can improve the performance for specific tasks.