• Hand-Tracking
• Computer Vision
• Machine Learning
• Optimization

HandFlow: Quantifying View-Dependent 3D Ambiguity in Two-Hand Reconstruction with Normalizing Flow J. Wang, D Luvizon, F. Mueller, F. Bernard, A. Kortylewski, D. Casas, C. Theobalt Vision, Modeling, and Visualization (VMV), 2022 (Best Paper Honorable Mention) This work presents the first probabilistic method to estimate a distribution of plausible two-hand poses given a monocular RGB input. It quantitatively shows that existing determinstic methods are not suited for this ambigious task, while the proposed method better captures the available information. [project page] [Arxiv]

EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream V. Rudnev, V. Golyanik, J. Wang, H.-P. Seidel, F. Mueller, M. Elgharib, C. Theobalt International Conference on Computer Vision (ICCV), 2021 This work presents the first hand tracking method to use event stream input. This allows the method to track fast hand motions at 1000 Hz while additionally dealing with low light environments. [project page] [pdf]

RGB2Hands: Real-Time Tracking of 3D Hand Interactions from Monocular RGB Video J. Wang, F. Mueller, F. Bernard, S. Sorli, O. Sotnychenko, N. Qian, M. A. Otaduy, D. Casas, C. Theobalt ACM Transactions on Graphics (Proc. of SIGGRAPH Asia 2020) In this work we present the first real-time method for motion capture of skeletal pose and 3D surface geometry of two hands from a single RGB camera that explicitly considers close interactions. [project page] [pdf] [video]

HTML: A Parametric Hand Texture Model for 3D Hand Reconstruction and Personalization N. Qian, J. Wang, F. Mueller, F. Bernard, V. Golyanik, C. Theobalt European Conference on Computer Vision (ECCV) 2020 In this work we present HTML, the first parametric texture model of human hands. Our model spans several dimensions of hand appearance variability (e.g., related to gender, ethnicity, or age) and only requires a commodity camera for data acquisition. Experimentally, we demonstrate that our appearance model can be used to tackle a range of challenging problems such as 3D hand reconstruction from a single monocular image. Furthermore, our appearance model can be used to define a neural rendering layer that enables training with a self-upervised photometric loss. We make our model publicly available. [project page] [pdf]

Generative Model-Based Loss to the Rescue: A Method to Overcome Annotation Errors for Depth-Based Hand Pose Estimation J. Wang, F. Mueller, F. Bernard, C. Theobalt International Conference on Automatic Face and Gesture Recognition (FG) 2020 In this paper we propose to use a model-based generative loss for training hand pose estimators on depth images based on a volumetric hand model. We demonstrate that such an approach can be used to train on datasets that have erroneous annotations while obtaining predictions that explain the depth images better than the annotions. [project page] pdf]

Convex Optimisation for Inverse Kinematics T. Yenamandra, F. Bernard, J. Wang, F. Mueller, C. Theobalt International Conference on 3D Vision (3DV) 2019 In this paper we propose a convex optimisation approach for the IK problem based on semidefinite programming, which admits a polynomial-time algorithm that globally solves (a relaxation of) the IK problem. Experimentally, we demonstrate that the proposed method significantly outperforms local optimisation methods using different real-world skeletons. [pdf]

• April 2021 - August 2021:
  Tutor For Computer Vision and Machine Learning for Computer Graphics Seminar, Organizers: Dr. Mohamed Elgharib, Dr. Vladislav Golyanik, Prof. Dr. Christian Theobalt, Saarland University
  
• April 2018 - August 2018:
  Tutor For 3D Shape Analysis Seminar, Organizers: Dr. Florian Bernard, Prof. Dr. Christian Theobalt, Saarland University
  
• April 2017 - October 2017:
  Tutor For Image Processing and Computer Vision, Lecturer: Prof. Dr. Joachim Weickert, Saarland University

• May 2018 - present:
  Ph. D. student in Computer Science at the Universität des Saarlandes, Saarbrücken, Germany and the Max-Planck-Institut für Informatik
  
• April 2016 - April 2018:
  Master Studies in Computer Science at the Universität des Saarlandes, Saarbrücken, Germany
  
• September 2010 - June 2015:
  Bachelor Studies in Engineering Science at University of Toronto, Canada
  Title of Bachelor's Thesis: High Expenditure Patient Classification From Electronic Medical Records (Supervisor: Prof. Quaid Morris)