3D Face Expression Estimation and Generation from 2D Image Based on a Physically Constraint Model

Manuscript received March 17, 1999. Manuscript revised July 19, 1999. † The authors are with the Faculty of Engineering, Room 1201, Seikei University, Musashino-shi, 180-8633 Japan. †† The author is with the Department of Computer Science University of Toronto, 6 King’s Colledge Road, Room 283, Toronto, Ontario, Canada. SUMMARY Muscle based face image synthesis is one of the most realistic approaches to the realization of a life-like agent in computers. A facial muscle model is composed of facial tissue elements and simulated muscles. In this model, forces are calculated effecting a facial tissue element by contraction of each muscle string, so the combination of each muscle contracting force decides a specific facial expression. This muscle parameter is determined on a trial and error basis by comparing the sample photograph and a generated image using our Muscle-Editor to generate a specific face image. In this paper, we propose the strategy of automatic estimation of facial muscle parameters from 2D markers’movements located on a face using a neural network. This corresponds to the non-realtime 3D facial motion capturing from 2D camera image under the physics based condition. key words: facial muscle model, neural network, facial image reconstruction, muscle parameter