Marker-less Facial Motion Capture based on the Parts Recognition

Abstract

A motion capture method is used to capture facial motion to create 3D animations and for recognizing facial expressions. Since the facial motion consists of non-rigid deformations of a skin, it is difficult to track a transition of a point on the face over time. Therefore, a number of methods based on markers have been proposed to solve this problem. However, since it is difficult to place the markers on a face or on an actual texture of the face, it is difficult to apply the marker-based capture methods. To overcome this problem, we propose a marker-less motion capture method for facial motions. Since the thickness of a skin varies in each facial part, the features of the motion of the each parts also vary. These features make the non-rigid tracking problem more difficult. To prevent the problem, we recognize five types of facial parts (nose, mouth, eye, cheek and obstacle) from 3D points of a face by using Random Forest algorithm. After the recognition of the facial parts, we track the motion of the each part by using a non-rigid registration algorithm based on the Gaussian Mixture Model. Since the motions of the each part are independently detected, we integrate the motions of the each part as 3D shape deformations for tracking the motions of the points on the whole face. We adopt a Free-Form Deformation technique which is based on the Radial Basis Function for the integration. This deformation method deforms 3D shapes seamlessly with pairs of key points: several numbers of points of a source face and the corresponding points of a target shape which are detected by the non-rigid registration algorithm. Finally, we represent the motion of the face as the deformation from the face of the initial frame to the others. In our results, we show that the proposed method enables us to detect the motion of the face more accurately.