Emotions : the art of communication applied to virtual actors

This thesis contributes to a multi-disciplinary research field aiming to build autonomous virtual actors. Emotions are a very active topic of research in many domains, such as biology, sociology, and neurology, due to their powerful influences on human behaviours. Computer science also takes an interest in emotion to create believable autonomous virtual characters. Much research focuses on mechanisms assessing the environment of the virtual actor and generating emotional evaluations to influence the agent’s behaviours. In contrast, this thesis looks at the representation of the durations of emotions, the interactions between emotions, and the influences on the visual speech of virtual actors. Following the review of studies of human facial expression, emotion theories, facial animation systems, and computational models of emotion, a Dynamic Emotion Representation (DER) model and an Emotionally Expressive Facial Animation System (EE-FAS) have been designed. The DER model enables programmers to represent a network of persisting states, where each state can influence the responses of others. An instance of this model is implemented in this thesis and represents three types of state, changing on different timescales. The EE-FAS takes as input a text with tags representing communicative acts and produces visual speech. The presence of the DER in the EE-FAS architecture provides coherence in the production of emotional facial expressions, diversity in the visual representations of communicative acts and variability in the animations produced from a tagged text. The EE-FAS distinguishes and produces emotional expressions due to emotional events and communicative acts synchronised with the speech. To extend the facial vocabulary of virtual actors, the EE-FAS displays different facial expressions corresponding to a single communicative act, according to the DER state. It also distinguishes contexts for displaying genuine and fake facial expressions. An experiment was designed to measure the meanings communicated by facial component actions. This experiment, based on videos produced by the EE-FAS, shows that emotional meanings are perceived from facial component actions, and that the meanings of facial component action combinations are combinations of the meanings communicated by individual components.