Facial animation with motion capture based on surface blending

This paper proposes a methodology to reconstruct 3D facial expressions with motion capture data. Featurepoint based facial animation provides easy control of expression usually by moving surface points using mathematical deformation. However it does not support the high quality of surface animation of the face where the feature points are not present. In this paper, we focus on animating a 3D facial model using only feature points, but keeping the high quality animation by using an expression databank obtained from surface scanning. Firstly, a facial expression databank is prepared by processing the raw laser-scanned human face data with a consistent parameterization technique. Secondly, sparse motion capture data is obtained using an optical tracking system. Thirdly, guided by the captured MPEG-4 feature point motions, we find the corresponding surface information in the existing examples in the databank by linear combination of them. The optimized blending weights are obtained implicitly by Genetic Algorithms (GA). Finally, the surface blending result is retargeted into the performer’s neutral facial mesh. Consequently, motions of the facial surface points are reconstructed.