In the framework of MPEG-4, one can include applications where virtual agents, utilizing both textual and multisensory data, including facial expressions and nonverbal speech, help systems become accustomed to the actual feelings of the user. Applications of this technology are expected in educational environments, virtual collaborative workplaces, communities, and interactive entertainment. Fa...

Synthesis of realistic facial animation for arbitrary speech is an important but difficult problem. The difficulties lie in the synchronization between lip motion and speech, articulation variation under different phonetic context, and expression variation in different speaking style. To solve these problems, we propose a visual speech synthesis system based on a fivestate, multi-stream HMM, wh...

A novel approach is presented for generating VRML animation sequences from Sign Language notation, based on MPEG-4 Face and Body Animation. Sign Language notation, in the well-known SignWriting system, is provided as input and initially converted to SWML (SignWriting Markup Language), an XML-based format that has recently been developed for the storage, indexing, and processing of SignWriting n...

This paper proposes a scripting approach to facial animations for embodied agents, in particular for X3D/VRML-based web agents. The use of a scripting technology to control facial expressions has the advantage that it results in highly flexible components; not only can scripts be re-used to define a wide range of facial animation scenarios, but they can also be applied with appropriate paramete...

A real-time speech-driven synthetic talking face provides an effective multimodal communication interface in distributed collaboration environments. Nonverbal gestures such as facial expressions are important to human communication and should be considered by speech-driven face animation systems. In this paper, we present a framework that systematically addresses facial deformation modeling, au...

Muscle-based models of the human face produce high quality animation but rely on recorded muscle activity signals or synthetic muscle signals that are often derived by trial and error. This paper presents a dynamic inversion of a muscle-based model (Lucero and Munhall, 1999) that permits the animation to be created from kinematic recordings of facial movements. Using a nonlinear optimizer (Powe...

Eye Motion has a fundamental role in verbal and non-verbal communication among people. It can express emotions and regulate the flow of conversation. For this reason, it is very important that these movements can be realistic in Computer Animation’s applications, such as virtual humans in general and game/movies characters. This paper presents a model for producing eye movements in synthetic ag...

This chapter presents our research on real-time speech-driven face animation. First, a visual representation, called Motion Unit (MU), for facial deformation is learned from a set of labeled face deformation data. A facial deformation can be approximated by a linear combination of MUs weighted by the corresponding MU parameters (MUPs), which are used as the visual features of facial deformation...

Human interfaces for computer graphics systems are now evolving towards multi-modal approach. Information gathered using visual, audio and motion capture systems are now becoming increasingly important within user-controlled virtual environments. This paper discusses real-time interaction with virtual world through the visual analysis of human facial features from video. The underlying approach...

In this paper, we present a 3D facial model compliant with MPEG-4 specifications; our aim was the realization of an animated model able to simulate in a rapid and believable manner the dynamics aspect of the human face. We have realized a Simple Facial Animation Engine (SFAE) where the 3D proprietary facial model has the look of a young woman: “Greta". Greta is the core of an MPEG-4 decoder and...