A | B | C | D | E | F 
 G | H | I | J | K | L | M 
 N | O | P | Q | R | S | T 
 U | V | W | X | Y | Z 
max planck institut
informatik
mpii logo Minerva of the Max Planck Society

Homepage

Zollh&oumlfer, Michael

Michael Zollhöfer

MPI Informatics
Saarland Informatics Campus
Building E1.4, Office 219
D-66123 Saarbrücken
Germany

Email: mzollhoef-at-mpi-inf.mpg.de
Phone: +49 681 9325 4019


Michael Zollhöfer works as a Post Doctoral Researcher in computer vision and computer graphics at the Graphics, Vision & Video Group headed by Prof. Christian Theobalt. He received his PhD from the University of Erlangen-Nuremberg for his work on real-time non-rigid reconstruction.

During his PhD, he developed a data-parallel variant of Gauss-Newton that exploits modern graphics hardware. This non-linear solver enables the real-time solution of a wide range of computer graphics and computer vision problems, such as non-rigid tracking, shading-based refinement, intrinsic decomposition, bundle adjustment, multi-projection mapping and facial reenactment. His GPU solver is also the back end of a domain specific optimization language. In addition, his research interests include: face tracking from monocular video, high-quality inverse rendering, real-time hand-object tracking and template-free deformable reconstruction.

He is reviewing for the top journals and conferences:
SIGGRAPH, SIGGRAPH Asia, CVPR, ICCV, EG, PAMI, TVCG, CGF, ...


Publications


Corrective 3D Reconstruction of Lips from Monocular Video

P. Garrido M. Zollhöfer C. Wu D. Bradley P. Perez T. Beeler C. Theobalt

ACM Transactions on Graphics 2016 (TOG)   —   SIGGRAPH ASIA 2016

We present a novel approach for fully automatic reconstruction of detailed and expressive lip shapes along with the dense geometry of the entire face, from just monocular RGB video.

[paper], [video], [additional video], [internet video result], [project page]

Model-Based Teeth Reconstruction

C. Wu D. Bradley P. Garrido M. Zollhöfer C. Theobalt M. Gross T. Beeler

ACM Transactions on Graphics 2016 (TOG)   —   SIGGRAPH ASIA 2016

We present the first approach for non-invasive reconstruction of teeth from just a few input photographs.

[paper], [project page]

FaceVR: Real-Time Facial Reenactment and Eye Gaze Control in Virtual Reality

J. Thies M. Zollhöfer M. Stamminger C.Theobalt M. Nießner

arXiv 2016

We introduce FaceVR, a novel method for gaze-aware facial reenactment in the Virtual Reality (VR) context.

[arXiv], [project page]

Real-time Halfway Domain Reconstruction of Motion and Geometry

L. Thies M. Zollhöfer C. Richardt C. Theobalt G. Greiner

Proc. of the International Conference on 3D Vision 2016   —   3DV 2016

We present a novel approach for real-time joint reconstruction of 3D scene motion and geometry from binocular stereo videos.

[paper], [supplemental], [video], [poster], [project page]

Real-time Joint Tracking of a Hand Manipulating an Object from RGB-D Input

S. Sridhar F. Mueller M. Zollhöfer D. Casas A. Oulasvirta C. Theobalt

Proc. of the European Conference on Computer Vision 2016   —   ECCV 2016

We propose a real-time solution for joint tracking of a hand and a manipulated object using only a single commodity RGB-D camera.

[paper], [supplemental], [video], [poster], [data], [project page]

VolumeDeform: Real-time Volumetric Non-rigid Reconstruction

M. Innmann M. Zollhöfer M. Nießner C. Theobalt M. Stamminger

Proc. of the European Conference on Computer Vision 2016   —   ECCV 2016

We present a novel approach for the reconstruction of dynamic geometric shapes using a single hand-held consumer-grade RGB-D sensor at real-time rates.

[paper], [supplemental], [video], [poster], [data], [project page]

Opt: A Domain Specific Language for Non-linear Least Squares Optimization in Graphics and Imaging

Z. DeVito M. Mara M. Zollhöfer G. Bernstein J. Ragan-Kelley C. Theobalt P. Hanrahan M. Fisher M. Nießner

arXiv 2016

We propose a new language, Opt, in which a user simply writes energy functions over image- or graph-structured unknowns, and a compiler automatically generates state-of-the-art GPU optimization kernels.

[arXiv], [project page]

Live Intrinsic Video

A. Meka M. Zollhöfer C. Richardt C. Theobalt

ACM Transactions on Graphics 2016 (TOG)   —   SIGGRAPH 2016

We present a novel variational approach to tackle the underconstrained intrinsic video decomposition problem at real-time frame rates enabling on-line processing of live video footage.

[pdf], [video], [additional video], [talk], [data], [project page]

BundleFusion: Real-time Globally Consistent 3D Reconstruction using Online Surface Re-integration

A. Dai M. Nießner M. Zollhöfer S. Izadi C.Theobalt

arXiv 2016

We propose a novel, real-time, end-to-end RGB-D reconstruction framework that has a global pose optimization strategy at its core.

[arXiv], [video], [data], [project page]

Face2Face: Real-time Face Capture and Reenactment of RGB Videos

J. Thies M. Zollhöfer M. Stamminger C. Theobalt M. Nießner

Proc. Computer Vision and Pattern Recognition 2016   —   CVPR 2016

We present a novel approach for real-time facial reenactment of a monocular target video sequence (e.g., Youtube video). The source sequence is also a monocular video stream, captured live with a commodity webcam.

[pdf], [supplemental], [video], [talk], [poster], [project page]

Reconstruction of Personalized 3D Face Rigs from Monocular Video

P. Garrido M. Zollhöfer D. Casas L. Valgaerts K. Varanasi P. Perez C. Theobalt

ACM Transactions on Graphics 2016 (TOG)   —   SIGGRAPH 2016

We present a novel approach for the automatic creation of a personalized high-quality 3D face rig of an actor from just monocular video data, e.g. vintage movies.

[pdf], [supplemental pdf], [video], [additional video], [talk], [project page]

Real-time Expression Transfer for Facial Reenactment

J. Thies M. Zollhöfer M. Nießner L. Valgaerts M. Stamminger C. Theobalt

ACM Transactions on Graphics 2015 (TOG)   —   SIGGRAPH ASIA 2015

We present a method for the real-time transfer of facial expressions from an actor in a source video to an actor in a target video, thus enabling the ad-hoc control of the facial expressions of the target actor.

[pdf], [video], [talk], [project page]

Real-Time Pixel Luminance Optimization for Dynamic Multi-Projection Mapping

C. Siegl M. Colaianni L. Thies J. Thies M. Zollhöfer S. Izadi M. Stamminger F. Bauer

ACM Transactions on Graphics 2015 (TOG)   —   SIGGRAPH ASIA 2015

We present a real-time solution for smooth blending across multiple projectors using a new optimization framework that simulates the diffuse direct light transport of the physical world.

[pdf], [video], [project page]

Real-Time Reconstruction of Static and Dynamic Scenes

M. Zollhöfer

PhD Thesis (Published by Dr. Hut)

We present new techniques and algorithms for building three-dimensional representations of arbitrary objects, manually editing the reconstructions and tracking their non-rigid motion at real-time rates.

[pdf], [talk], [project page]

Shading-based Refinement on Volumetric Signed Distance Functions

M. Zollhöfer A. Dai M. Innmann C. Wu M. Stamminger C.Theobalt M. Nießner

ACM Transactions on Graphics 2015 (TOG)   —   SIGGRAPH 2015

We present a novel method to obtain fine-scale detail in 3D reconstructions generated with low-budget RGB-D cameras or other commodity scanning devices.

[pdf], [video], [talk], [data], [project page]

Low-Cost Real-Time 3D Reconstruction of Large-Scale Excavation Sites

M. Zollhöfer, C. Siegl, M. Vetter, B. Dreyer, M. Stamminger, S. Aybek, F. Bauer

Journal on Computing and Cultural Heritage 2015   —   JOCCH 2015

The 3D reconstruction of archeological sites is expensive and a time-consuming task. In this article, we present a novel approach to 3D reconstruction and compare to a standard photogrammetry pipeline.

[pdf], [video], [project page]

Real-time Shading-based Refinement for Consumer Depth Cameras

C. Wu, M. Zollhöfer, M. Nießner, M. Stamminger, S. Izadi, C.Theobalt

ACM Transactions on Graphics 2014 (TOG)   —   SIGGRAPH ASIA 2014

We present the first real-time method for refinement of depth data using shape-from-shading in general uncontrolled scenes.

[pdf], [video], [talk], [project page]

Interactive Model-based Reconstruction of the Human Head
using an RGB-D Sensor


M. Zollhöfer, J. Thies, M. Colaianni, M. Stamminger, G. Greiner

Journal Computer Animation and Virtual Worlds 2014   —   CASA 2014

We present a novel method for the interactive markerless reconstruction of human heads using a single commodity RGB-D sensor.

[pdf], [video], [talk], [project page]

Real-time Non-rigid Reconstruction using an RGB-D Camera

M. Zollhöfer, M. Nießner, S. Izadi, C. Rehmann, C. Zach, M. Fisher, C. Wu, A. Fitzgibbon,
C. Loop, C.Theobalt, M. Stamminger

ACM Transactions on Graphics 2014 (TOG)   —   SIGGRAPH 2014

We present a combined hardware and software solution for markerless reconstruction of non-rigidly deforming physical objects with arbitrary shape in real-time.

[pdf], [supplemental], [video], [talk], [project page]

Low-Cost Real-Time 3D Reconstruction of Large-Scale
Excavation Sites using an RGB-D Camera


M. Zollhöfer, B. Riffelmacher, M. Vetter, B. Dreyer, M. Stamminger, F. Bauer

EUROGRAPHICS Workshop on Graphics and Cultural Heritage 2014   —   GCH 2014

In this paper, we present an end-to-end pipeline for the online reconstruction of large-scale excavation sites using an RGB-D camera.

[pdf], [video], [talk], [project page]

Real-time 3D Reconstruction at Scale using Voxel Hashing

M. Nießner, M. Zollhöfer, S. Izadi, M. Stamminger

ACM Transactions on Graphics 2013 (TOG)   —   SIGGRAPH ASIA 2013

We contribute an online system for large and fine scale volumetric reconstruction based on a memory and speed efficient data structure.

[pdf], [video], [short video], [talk], [code], [project page]

Pseudo-Skeleton based ARAP Mesh Deformation

M. Zollhöfer, A. Vieweg, J. Süßmuth, G. Greiner

Proceedings of Computer Animation and Social Agents 2013 (Short Papers)   —   CASA 2013

In this work, we present a handle-based direct manipulation paradigm that builds upon a pseudo-skeleton with bones that are allowed to stretch and twist.

[pdf], [video], [talk], [project page]

Visualization and Deformation Techniques for Entertainment
and Training in Cultural Heritage


M. Zollhöfer, J. Süßmuth, F. Bauer, M. Stamminger, M. Boss

Image and Signal Processing and Analysis 2013   —   ISPA 2013

We think that state-of-the-art techniques in computer graphics and geometry processing can be leveraged in training and entertainment to make the topic of cultural heritage more accessible to a wider audience.

[pdf], [video], [talk], [project page]

GPU based ARAP Deformation using Volumetric Lattices

M. Zollhöfer, E. Sert, G. Greiner, J. Süßmuth

Eurographics 2012 (Short Papers)   —   EG 2012

We present a novel lattice based direct manipulation paradigm (LARAP) for mesh editing that decouples the runtime complexity from the mesh’s geometric complexity.

[pdf], [video], [talk], [project page]

Automatic Reconstruction of Personalized Avatars from 3D Face Scans

M. Zollhöfer, M. Martinek, G. Greiner, M. Stamminger, J. Süßmuth

Journal Computer Animation and Virtual Worlds 2011   —   CASA 2011

We present a simple algorithm for computing a high quality personalized avatar from a single color image and the corresponding depth map which have been captured by Microsoft’s Kinect sensor.

[pdf], [video], [talk], [project page]

Meshless Hierarchical Radiosity on the GPU

M. Zollhöfer, M. Stamminger

Vision, Modeling, and Visualization 2011   —   VMV 2011

In this paper, we analyze the bottlenecks of meshless radiosity and examine the possibilities for an efficient and parallel implementation on the GPU.

[pdf], [video], [talk], [poster], [project page]

Animation Transplantation

J. Süßmuth, M. Zollhöfer, G. Greiner

Journal Computer Animation and Virtual Worlds 2010   —   CASA 2010

We present a novel method to animate a static geometry by cloning a captured animation sequence.

[pdf], [video], [project page]


Exhibits/Demos


FaceInCar: Real-time Dense Monocular Face Tracking of a Driver

M. Zollhöfer J. Thies M. Nießner M. Stamminger C. Theobalt

National IT Summit 2016

This exhibit demonstrates the first approach for real-time dense face tracking of a driver based on a monocular video stream.

[project page]

Demo of Face2Face: Real-time Face Capture and Reenactment of RGB Videos

J. Thies M. Zollhöfer M. Stamminger C. Theobalt M. Nießner

SIGGRAPH 2016 (Emerging Technologies)

This live demonstration of the Face2Face approach allows for real-time facial reenactment of a monocular target video sequence (e.g., Youtube video).

[project page]