Hand-over-face gestures, a subset of emotional body language, are overlooked by automatic affect inference systems. We propose the use of hand-over-face gestures as a novel affect cue for automatic inference of cognitive mental states. Moreover, affect recognition systems rely on the existence of publicly available datasets, often the approach is only as good as the data. We present the collect...

We present a high level formalism for specifying verbal and nonverbal output from a multimodal dialogue system. The output specification is XML-based and provides information about communicative functions of the output without detailing the realisation of these functions. The specification can be used to control an animated character that uses speech and gestures. We give examples from an imple...

Facial expressions play an important role in interpersonal relations as well as for security purposes. The malicious intensions of a thief can be recognized with the help of his gestures, facial expressions being its major part. This is because humans demonstrate and convey a lot of evident information visually rather than verbally. Although humans recognize facial expressions virtually without...

We present a high level formalism for specifying verbal and nonverbal output from a multimodal dialogue system. The output specification is XML-based and provides information about communicative functions of the output without detailing the realisation of these functions. The specification can be used to control an animated character that uses speech and gestures. We give examples from an imple...

This paper describes a method for acquiring data for facial movements to be analysed for implementation in an animated talking head. We will show preliminary data on how a number of articulatory parameters vary under the influence of expressiveness in speech and gestures. Primarily we focused on expressive gestures and emotions conveying information that is intended to make the animated agent m...

Verbal data is the primary focus for analysis in the prevalent Usability evaluations like in ‘Think Aloud Method’. This study involves 18 cross cultural TA tests and it was found that users use gestures profoundly to communicate their mental activities. It was observed that hand gestures are attempts to communicate abstract feelings as well as to quantify, to simplify a complex expression & ref...

This paper presents results from two perception experiments designed to investigate intonational cues and visual facial cues to interrogative mode in Swedish. Results from the intonation test indicate that both a widened F0 range on a final focal accent and time alignment properties of the F0 rise and peak make important contributions to the interrogative percept. Results from the audiovisual t...

We present a series of experiments investigating face-to-face interaction between an Embodied Conversational Agent (ECA) and a human interlocutor. The ECA is embodied by a video realistic talking head with independent head and eye movements. For a beneficial application in face-to-face interaction, the ECA should be able to derive meaning from communicational gestures of a human interlocutor, a...

With the development of new trends in human-machine interfaces, animated feature films and video games, better avatars and virtual agents are required that more accurately mimic how humans communicate and interact. Gestures and speech are jointly used to express intended messages. The tone and energy of the speech, facial expression, rigid head motion and hand motion combine in a non-trivial ma...

Human emotions are expressed using the facial expressions, the tone of voice, hands and body gestures .An approach of interaction between the computer-human must be accurate and robust. We use the concept of multi classifier systems to study the human emotions, audio-visual detection system. Automate recognition of emotional state, machines must be taught expressions to understand facial gestur...