Modality Theory in Support of Multimodal Interface Design

Modality Theory is a comparatively new field of investigation which addresses the following general problem of mapping task domain information into interactive multimodal interfaces: given any particular set of information which needs to be exchanged between user and system during task performance in context, identify the input/ output modalities which constitute an optimal solution to the representation and exchange of that information. This paper proposes a research agenda for Modality Theory and presents two steps towards its implementation. The first is a generative taxonomy of output modalities covering the media of graphics, sound and touch. The second is a methodology for carrying out information-mapping in design practice. As it matures, Modality Theory promises to provide useful support to contemporary designers of interactive human-computer interfaces who have begun to use a rapidly increasing number of different, and often alternative input/output modalities for the expression and exchange of information between systems and their users.