office: |
Campus E1 4,
Room 115E Saarland Informatics Campus 66123 Saarbrücken Germany |
---|---|
email: |
Get my email address via email |
phone: | +49 681 9325 4549 |
State of the Art in Dense Monocular Non-Rigid 3D Reconstruction
Edith Tretschk*,
Navami Kairanda*,
Mallikarjun B R,
Rishabh Dabral,
Adam Kortylewski,
Bernhard Egger,
Marc Habermann,
Pascal Fua,
Christian Theobalt,
Vladislav Golyanik
(* equal contribution) Eurographics 2023 (STAR Report) This survey focuses on state-of-the-art methods for dense non-rigid 3D reconstruction of various deformable objects and composite scenes from monocular videos or sets of monocular views. It reviews the fundamentals of 3D reconstruction from 2D image observations. We then start from general methods, and proceed towards techniques making stronger assumptions about the observed objects (e.g. human faces, bodies, hands, and animals). We conclude by discussing open challenges in the field and the social aspects associated with the usage of the reviewed methods. [arXiv] [project page] |
φ-SfT: Shape-from-Template with a Physics-based Deformation Model Navami Kairanda, Edith Tretschk, Mohamed Elgharib, Christian Theobalt, Vladislav Golyanik Computer Vision and Pattern Recognition (CVPR), 2022. We propose a new SfT approach explaining the observations through simulation of a physically-based surface deformation model representing forces and material properties. In contrast to previous works, we utilise a differentiable physics-based simulator to regularise the surface evolution. In addition, we regress the material properties such as its bending coefficients, elasticity, stiffness, and material density. For the evaluation, we record with an RGB-D camera challenging real surfaces with various material properties and texture, exposed to physical forces. Our approach reconstructs the underlying deformations much more accurately than related methods. [paper][project page (incl. code & data)] |