Mobile interfaces need to allow the user and system to adapt their choice of communication modes according to user preferences, the task at hand, and the physical and social environment. We describe a multimodal application architecture which combines finite-state multimodal language processing, a speech-act based multimodal dialogue manager, dynamic multimodal output generation, and user-tailo...

A widely accepted prediction is that computing will move to the background, weaving itself into the fabric of our everyday living spaces and projecting the human user into the foreground. To realise this prediction, next-generation computing should develop anticipatory user interfaces that are human-centred, built for humans and based on naturally occurring multimodal human communication. These...

Closeups are used in illustrations to provide detailed views on regions of interest. They are integrated into the rendering of the whole structure in order to reveal their spatial context. In this paper we present the concept of interactive closeups for medical reporting. Each closeup is associated with a region of interest and may show a single modality or a desired combination of the availabl...

Our capacity to engage in meaningful conversations depends on a multitude of communication cues: verbal delivery of articulate and intelligible speech, tone and modulation of voice, exhibition of a range of facial expressions, and display of body gestures, among others.Parkinson’s disease diminishes verbal and non-verbal communication facilities in affected persons. Occupational therapists meas...

In this paper, we describe TYCOON, a model that we are developing for observing, evaluating and specifying cooperations between software and human agents. This model is based on a typology made of six primitive types of cooperations: equivalence, specialization, transfer, redundancy, complementarity and concurrency. Each of these types may be involved in several goals of cooperation such as ena...

We describe a novel framework to detect ball hits in a tennis game by combining audio and visual information. Ball hit detection is a key step in understanding a game such as tennis, but single-mode approaches are not very successful: audio detection suffers from interfering noise and acoustic mismatch, video detection is made difficult by the small size of the ball and the complex background o...