Franziska Mueller

Franziska Mueller

Max-Planck-Institut für Informatik
Department 4: Computer Graphics
 office: Campus E1 4, Room 211 C
Saarland Informatics Campus
66123 Saarbrücken


 email: Get my email address via email
 phone: +49 681 9325 4057
I'm a 4th year Ph.D. student in the Graphics, Vision and Video group supervised by Prof. Dr. Christian Theobalt.

Research Interests

  • Real-time tracking of interacting hands
  • Model- and optimization-based reconstruction of articulated motion
  • Machine learning techniques for computer vision


XNect: Real-time Multi-person 3D Human Pose Estimation with a Single RGB Camera

D. Mehta, O. Sotnychenko, F. Mueller, W. Xu, M. Elgharib, P. Fua, H.P. Seidel, H. Rhodin, G. Pons-Moll, C. Theobalt

A real-time multi-person 3D human body pose estimation system requiring only a single RGB camera for human motion capture in general scenes. The system produces full body 3D articulation estimates even under strong partial occlusion, as well as estimates of camera relative localization in space. The system can handle an arbitrary number of people in the scene, and processes complete frames without requiring prior person detection. A novel efficient convolutional network architecture (SelecSLS) enables real-time performance without compromising on accuracy. The pose formulation has the fully convolutional network only reason about the visible body parts, and occluded part reasoning is offloaded to a subsequent small fully connected network which gathers the full body context.

[arXiv] [Demo@CVPR19] [SelecSLS-Pytorch]

Convex Optimisation for Inverse Kinematics

T. Yenamandra, F. Bernard, J. Wang, F. Mueller, C. Theobalt

International Conference on 3D Vision (3DV) 2019

In this paper we propose a convex optimisation approach for the IK problem based on semidefinite programming, which admits a polynomial-time algorithm that globally solves (a relaxation of) the IK problem. Experimentally, we demonstrate that the proposed method significantly outperforms local optimisation methods using different real-world skeletons.


Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Camera

F. Mueller, M. Davis, F. Bernard, O. Sotnychenko, M. Verschoor, M. A. Otaduy, D. Casas, C. Theobalt


We present a novel method for real-time pose and shape reconstruction of two strongly interacting hands. Our approach is the first two-hand tracking solution that combines an extensive list of favorable properties, namely it is marker-less, uses a single consumer-level depth camera, runs in real time, handles inter- and intra-hand collisions, and automatically adjusts to the user's hand shape.

[pdf] [video] [project page]

HandSeg: An Automatically Labeled Dataset for Hand Segmentation from Depth Images

A. Bojja, F. Mueller, S. Malireddi, M. Oberweger, V. Lepetit, C. Theobalt, K. Yi, A. Tagliasacchi

Conference on Computer and Robot Vision (CRV) 2019

We propose an automatic method for generating high-quality annotations for depth-based hand segmentation, and introduce a large-scale hand segmentation dataset

[arxiv] [project page]

FingerInput: Capturing Expressive Single-Hand Thumb-to-Finger Microgestures for On-Skin Input

M. Soliman, F. Mueller, L. Hegemann, J. Sol Roo, C. Theobalt, J. Steimle

ACM International Conference on Interactive Surfaces and Spaces (ISS) 2018
Best Academic Paper Award

In this paper, we present a consolidated design space for thumb-to-finger microgestures. Based on this design space, we introduce a thumb-to-finger gesture recognition system which achieves an average accuracy of 91% for real-time detection of 8 demanding gesture classes.

[pdf] [project page]

Single-Shot Multi-Person 3D Body Pose Estimation From Monocular RGB Input

D. Mehta, O. Sotnychenko, F. Mueller, W. Xu, S. Sridhar, G. Pons-Moll, C. Theobalt

International Conference on 3D Vision (3DV) 2018

A single-shot approach to jointly predict 3D body pose of multiple subjects in general scenes without requiring prior bounding-box extraction. Trained on new composited MuCo-3DHP dataset and evaluated on a new recorded multi-person 3D pose benchmark MuPoTS-3DHP.

[arxiv] [project page]

GANerated Hands for Real-Time 3D Hand Tracking from Monocular RGB

F. Mueller, F. Bernard, O. Sotnychenko, D. Mehta, S. Sridhar, D. Casas, C. Theobalt

Spotlight @ Computer Vision and Pattern Recognition (CVPR) 2018

We propose a novel method for real-time hand tracking from monocular RGB input which combines a CNN-based regressor and a kinematic optimization framework. To enhance our synthetic training data, we introduce a new geometrically consistent image-to-image translator for unpaired examples.

[pdf] [supplemental] [video] [spotlight presentation] [project page]

Real-time Hand Tracking under Occlusion from an Egocentric RGB-D Sensor

F. Mueller, D. Mehta, O. Sotnychenko, S. Sridhar, D. Casas, C. Theobalt

International Conference on Computer Vision (ICCV) 2017

We present a method for real-time hand tracking under occlusion in cluttered egocentric scenes from a single RGB-D camera. To enable training of our machine learning components, we introduce a new large-scale dataset SynthHands which was captured using a mixed reality approach. Furthermore, we propose a real benchmark dataset EgoDexter which provides annotated fingertip positions.

[pdf] [supplemental] [video] [project page]

Real-time Joint Tracking of a Hand Manipulating an Object from RGB-D Input

S. Sridhar, F. Mueller, M. Zollhöfer, D. Casas, A. Oulasvirta, C. Theobalt

European Conference on Computer Vision (ECCV) 2016

In this work we tackle the especially challenging problem of tracking joint hand and object motion while running at realtime framerates.

[pdf] [supplemental] [poster] [video] [project page] [Dexter+Object dataset]

Fast and Robust Hand Tracking Using Detection-Guided Optimization

S. Sridhar, F. Mueller, A. Oulasvirta, C. Theobalt

Computer Vision and Pattern Recognition (CVPR) 2015

We present a novel approach for real-time tracking of hand motion from a single commodity depth camera. The method combines a generative pose optimization framework with discriminative part label evidence for robustness and recovery.

[pdf] [supplemental] [poster] [video] [project page]

Awards & Honors


  • Summer semester 2019:
    Seminar Computer Vision and Machine Learning for Computer Graphics, Saarland University and MPI for Informatics
  • Summer semester 2018:
    Seminar 3D Shape Analysis, Saarland University and MPI for Informatics
  • Summer semester 2016, 2017:
    Seminar Computer Vision for Computer Graphics, Saarland University and MPI for Informatics
  • September/October 2015:
    Voluntary lecturer and coach for the Mathematik-Vorkurs für Informatiker (Math preparation course for new CS students), Saarland University
  • Winter semester 2014/15:
    Tutor for Grundzüge der Theoretischen Informatik (Theoretical Computer Science), Lecturer: Prof. Dr. Markus Bläser, Saarland University
  • Summer semester 2014:
    Tutor for Systemarchitektur (System Architecture), Lecturer: Prof. Dr. Jan Reineke, Saarland University
  • Winter semester 2013/14:
    Tutor for Programmierung 1 (Programming 1), Lecturer: Prof. Dr. Holger Hermanns, Saarland University
  • March 2013:
    Tutor for re-exam preparation in Programmierung 1 (Programming 1), Lecturer: Prof. Dr. Gert Smolka, Saarland University





  • Karnevalistischer Tanzsport (Carnival Dancing), a probably really German thing (with the awesome folks here or here on facebook). You can check out our most recent dance here.
  • Calisthenics
  • Reading (mostly fantasy)