Homepage

    Contact

    Firstname Lastname

    Marc Habermann

    Max-Planck-Institut für Informatik
    Department 4: Computer Graphics
    Graphics, Vision and Video
     office: Campus E1 4, Room 224
    Saarland Informatics Campus
    66123 Saarbrücken
    Germany
     email: mhaberma@mpi-inf.mpg.de
     phone: +49 681 9325-4024
     fax: +49 681 9325-4099

    Research Interests

    • Computer Vision and Computer Graphics

    • Reconstruction of Non-Rigid Deformations from RGB Video

    • Texture-Based Descriptors

    Publications

    ReTiCaM: Real-time Human Performance Capture from Monocular Video

    Marc Habermann   Weipeng Xu   Michael Zollhoefer   Gerard Pons-Moll   Christian Theobalt

    arxiv 2018

    We present the first real-time human performance capture approach that reconstructs dense, space-time coherent deforming geometry of entire humans in general everyday clothing from just a single RGB video.We propose a novel two-stage analysis-by-synthesis optimization whose formulation and implementation are designed for high performance. In the first stage, a skinned template model is jointly fitted to background subtracted input video, 2D and 3D skeleton joint positions found using a deep neural network, and a set of sparse facial landmark detections. In the second stage, dense non-rigid 3D deformations of skin and even loose apparel are captured based on a novel real-time capable algorithm for non-rigid tracking using dense photometric and silhouette constraints. Our novel energy formulation leverages automatically identified material regions on the template to model the differing non-rigid deformation behavior of skin and apparel. The two resulting nonlinear optimization problems per-frame are solved with specially-tailored data-parallel Gauss-Newton solvers. In order to achieve real-time performance of over 25Hz, we design a pipelined parallel architecture using the CPU and two commodity GPUs. Our method is the first real-time monocular approach for full-body performance capture. Our method yields comparable accuracy with off-line performance capture techniques, while being orders of magnitude faster.

    [pdf], [video], [project page]



    Neural Animation and Reenactment of Human Actor Videos

    Lingjie Liu   Weipeng Xu   Michael Zollhoefer   Hyeongwoo Kim   Florian Bernard   Marc Habermann   Wenping Wang   Christian Theobalt

    arxiv 2018

    We propose a method for generating (near) video-realistic animations of real humans under user control. In contrast to conventional human character rendering, we do not require the availability of a production-quality photo-realistic 3D model of the human, but instead rely on a video sequence in conjunction with a (medium-quality) controllable 3D template model of the person. With that, our approach significantly reduces production cost compared to conventional rendering approaches based on production-quality 3D models, and can also be used to realistically edit existing videos. Technically, this is achieved by training a neural network that translates simple synthetic images of a human character into realistic imagery. For training our networks, we first track the 3D motion of the person in the video using the template model, and subsequently generate a synthetically rendered version of the video. These images are then used to train a conditional generative adversarial network that translates synthetic images of the 3D model into realistic imagery of the human. We evaluate our method for the reenactment of another person that is tracked in order to obtain the motion data, and show video results generated from artist-designed skeleton motion. Our results outperform the state-of-the-art in learning-based human image synthesis.

    [pdf], [video], [project page]



    NRST: Non-rigid Surface Tracking from Monocular Video

    Marc Habermann   Weipeng Xu   Helge Rhodin   Michael Zollhoefer   Gerard Pons-Moll   Christian Theobalt

    Oral @ German Conference on Pattern Recognition (GCPR) 2018

    We propose an efficient method for non-rigid surface tracking from monocular RGB videos. Given a video and a template mesh, our algorithm sequentially registers the template non-rigidly to each frame.We formulate the per-frame registration as an optimization problem that includes a novel texture term specifically tailored towards tracking objects with uniform texture but fine-scale structure, such as the regular micro-structural patterns of fabric. Our texture term exploits the orientation information in the micro-structures of the objects, e.g., the yarn patterns of fabrics. This enables us to accurately track uniformly colored materials that have these high frequency micro-structures, for which traditional photometric terms are usually less effective. The results demonstrate the effectiveness of our method on both general textured non-rigid objects and monochromatic fabrics.

    [pdf], [video], [project page]



    Teaching

    • September 2016 - June 2018
      Supervisor for 3D Shape Analysis, Lecturer: Dr. Florian Bernard and Prof. Dr. Christian Theobalt at the Saarland University, Saarbrücken, Germany

    • September 2016 - June 2018
      Tutor for Seminarfach 3D Modellierung at the Leibniz Gymnasium/Albertus Magnus Gymnasium, Sankt Ingbert, Germany

    • July 2013 - September 2016:
      Tutor for 3D Modellierung Alte Schmelz, Sankt Ingbert, Germany

    Education

    • November 2017
      Günter-Hotz-Medal for the best Master graduates in Computer Science, Saarbrücken, Germany

    • September 2017 - present
      PhD student at the Max Planck Institute for Informatics in the GVV Group, Saarbrücken, Germany

    • April 2016 - November 2017
      Master Studies in Computer Science at Saarland University, Saarbrücken, Germany
      Title of Master's Thesis (Diplomarbeit): RONDA - Reconstruction of Non-rigid Surfaces from High Resolution Video (supervisor: Prof. Dr. Christian Theobalt) (PDF)

    • October 2012 - April 2016:
      Bachelor Studies in Computer Science at Saarland University, Saarbrücken, Germany
      Title of Bachelor's Thesis: Drone Path Planning (supervisor: Dr.-Ing. Tobias Ritschel) (PDF)

    • July 2012:
      Abitur at the Albertus Magnus Gymnasium, Sankt Ingbert, Germany

    Private

    Hobbies

    • Photography

    • Bouldering

    • Reading Books