Jiayi Wang

Jiayi Wang

Max-Planck-Institut für Informatik
D6: Visual Computing and Artificial Intelligence
 office: Campus E1 4,
Saarland Informatics Campus
66123 Saarbrücken
 email: jwang [at] mpi-inf dot mpg dot de
 phone: +49 681 9325-4057
 fax: +49 681 9325 4099

Research Interests

  • Hand-Tracking
  • Computer Vision
  • Machine Learning
  • Optimization


HandFlow: Quantifying View-Dependent 3D Ambiguity in Two-Hand Reconstruction with Normalizing Flow

J. Wang, D Luvizon, F. Mueller, F. Bernard, A. Kortylewski, D. Casas, C. Theobalt

Vision, Modeling, and Visualization (VMV), 2022 (Best Paper Honorable Mention)

This work presents the first probabilistic method to estimate a distribution of plausible two-hand poses given a monocular RGB input. It quantitatively shows that existing determinstic methods are not suited for this ambigious task, while the proposed method better captures the available information.

[project page] [Arxiv]

EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream

V. Rudnev, V. Golyanik, J. Wang, H.-P. Seidel, F. Mueller, M. Elgharib, C. Theobalt

International Conference on Computer Vision (ICCV), 2021

This work presents the first hand tracking method to use event stream input. This allows the method to track fast hand motions at 1000 Hz while additionally dealing with low light environments.

[project page] [pdf]

RGB2Hands: Real-Time Tracking of 3D Hand Interactions from Monocular RGB Video

J. Wang, F. Mueller, F. Bernard, S. Sorli, O. Sotnychenko, N. Qian, M. A. Otaduy, D. Casas, C. Theobalt

ACM Transactions on Graphics (Proc. of SIGGRAPH Asia 2020)

In this work we present the first real-time method for motion capture of skeletal pose and 3D surface geometry of two hands from a single RGB camera that explicitly considers close interactions.

[project page] [pdf] [video]

HTML: A Parametric Hand Texture Model for 3D Hand Reconstruction and Personalization

N. Qian, J. Wang, F. Mueller, F. Bernard, V. Golyanik, C. Theobalt

European Conference on Computer Vision (ECCV) 2020

In this work we present HTML, the first parametric texture model of human hands. Our model spans several dimensions of hand appearance variability (e.g., related to gender, ethnicity, or age) and only requires a commodity camera for data acquisition. Experimentally, we demonstrate that our appearance model can be used to tackle a range of challenging problems such as 3D hand reconstruction from a single monocular image. Furthermore, our appearance model can be used to define a neural rendering layer that enables training with a self-upervised photometric loss. We make our model publicly available.

[project page] [pdf]

Generative Model-Based Loss to the Rescue: A Method to Overcome Annotation Errors for Depth-Based Hand Pose Estimation

J. Wang, F. Mueller, F. Bernard, C. Theobalt

International Conference on Automatic Face and Gesture Recognition (FG) 2020

In this paper we propose to use a model-based generative loss for training hand pose estimators on depth images based on a volumetric hand model. We demonstrate that such an approach can be used to train on datasets that have erroneous annotations while obtaining predictions that explain the depth images better than the annotions.

[project page] pdf]

Convex Optimisation for Inverse Kinematics

T. Yenamandra, F. Bernard, J. Wang, F. Mueller, C. Theobalt

International Conference on 3D Vision (3DV) 2019

In this paper we propose a convex optimisation approach for the IK problem based on semidefinite programming, which admits a polynomial-time algorithm that globally solves (a relaxation of) the IK problem. Experimentally, we demonstrate that the proposed method significantly outperforms local optimisation methods using different real-world skeletons.



  • April 2021 - August 2021:
    Tutor For Computer Vision and Machine Learning for Computer Graphics Seminar, Organizers: Dr. Mohamed Elgharib, Dr. Vladislav Golyanik, Prof. Dr. Christian Theobalt, Saarland University
  • April 2018 - August 2018:
    Tutor For 3D Shape Analysis Seminar, Organizers: Dr. Florian Bernard, Prof. Dr. Christian Theobalt, Saarland University
  • April 2017 - October 2017:
    Tutor For Image Processing and Computer Vision, Lecturer: Prof. Dr. Joachim Weickert, Saarland University