Enhancing a WIMP based interface with speech, gaze tracking and agents

This paper describes an attempt to enhance a windows based (WIMP Windows Icon Menu Pointer) environment. The goal is to establish whether user interaction on the common desktop PC can be augmented by adding new modalities to the WIMP interface, thus bridging the gap between todays interaction patterns and future interfaces comprising e.g. advanced conversational capabilities, VR technology, etc. A user survey was carried out to establish the trouble spots of the WIMP interface on the most common desktop work station, the Windows 95 PC. On the basis of this, a number of new modalities were considered. Spoken inand output and gaze tracking were selected together with the concept of an interface agent for further investigation. A system was developed to control the interaction of the inand output modalities, and set of five scenarios were constructed to test the proposed ideas. In these, a number of test subjects used the existing and added modalities in various configurations.