A | B | C | D | E | F 
 G | H | I | J | K | L | M 
 N | O | P | Q | R | S | T 
 U | V | W | X | Y | Z 
max planck institut informatik
mpii logoMinerva of the Max Planck Society

Marker-less Deformable Mesh Tracking for Human Shape and Motion Capture.



Primary Investigators: E. de Aguiar, C. Theobalt and C. Stoll

One of the most important and most difficult tasks in 3D Video processing of human actors and other moving subjects, is the reconstruction of the complex time-varying surface representation and its motion from the input camera views. In order to deal with this challenging problem, we developed an approach that uses a high-quality scan of a subject as shape model, and captures its motion from multi-view video by means of a fast Laplacian deformation approach and an image-based 3D correspondence estimation method. [1]. Without using optical markers and not relying on kinematic skeletons, as traditional motion capture methods, our algorithm can handle humans wearing arbitrary clothing, including wide t-shirts, skirts and even kimonos. The subject's overall motion, as well as non-rigid surface deformations are captured by the same unifying framework. Furthermore, our algorithm can straightforwardly be applied to any type of subject for which a scan exists, e.g. animals. Since it is based on a static template, it preserves the connectivity of the mesh over time, which is particularly important when it comes to our envisioned 3D Video applications and dynamic shape encoding. Being easy to implement, our algorithm is able to make a laser scan of a subject move and deform in the same way as its real-world counterpart in video. We demonstrated the performance and accuracy of the method using synthetic and captured real-world video sequences. To our knowledge, this is the first system in the literature that can passively capture people with arbitrarily complex apparel, as well as other subjects exhibiting complex surface deformation.

Videos:

[Video showing the performance of our approach]

References:

[1] E. de Aguiar, C. Theobalt, C. Stoll and H.-P. Seidel, Marker-less Deformable Mesh Tracking for Human Shape and Motion Capture. In Proc. of IEEE CVPR 2007, Minneapolis, USA. [pdf]

@INPROCEEDINGS{deAguiarCVPR2007,
AUTHOR = {de Aguiar, Edilson and Theobalt, Christian and Stoll, Carsten and Seidel, Hans-Peter},
TITLE = {Marker-less Deformable Mesh Tracking for Human Shape and Motion Capture},
BOOKTITLE = {IEEE International Conference on Computer Vision and Pattern Recognition (CVPR)},
PUBLISHER = {IEEE},
YEAR = {2007},
ORGANIZATION = {IEEE},
PAGES = {XX--XX},
ADDRESS = {Minneapolis, USA},
MONTH = {June},
}

Search MPII (type ? for help)