Consolidating Segmentwise Non-Rigid Structure from Motion

Abstract

This paper introduces a new segmentwise technique which consolidates multiple principles for nonrigid structure from motion (NRSfM) into a single energy-based framework. The energy functional of our Consolidating Monocular Dynamic Reconstruction (CMDR) approach is optimised by non-linear least squares and includes terms allowing to define the deformation model and additional constraints simultaneously in the metric and trajectory spaces. The proposed method achieves high accuracy on several tested sequences while providing robustness and scalability due to the spatial scene segmentation and the new lifted spatial Laplacian term. CMDR is flexible and easy to implement, thanks to the unified optimisation framework. It allows for scenario-specific extensions and can be used for rapid prototyping of new NRSfM methods.