• 12.08.2022: [New] MoCapDeform received Best Student Paper Award at 3DV 2022.
• 05.08.2022: [New] A paper on global 3D human motion capture (MoCapDeform) accepted at 3DV 2022.
• 25.07.2022: [New] 4DQV will present several papers at ECCV 2022:
  • UnrealEgo: A New Dataset for Robust Egocentric 3D Human Motion Capture.
  • Quantum Motion Segmentation.
  • HULC: 3D HUman Motion Capture with Pose Manifold Sampling and Dense Contact Guidance.
  • Neural Radiance Fields for Outdoor Scene Relighting.
• 04.07.2022: [New] The source code of Physical Inertial Poser (PIP) is now available here: github link.
• 27.06.2022: [New] The source code of φ-SfT ist now available here: github link.
• 21.04.2022: [New] The source code of Neural PhysCap ist now available via github.
• 19.02.2022: We are co-organising a tutorial associated with our STAR on Advances in Neural Rendering on the 27th of April at EUROGRAPHICS'22.
• 28.03.2022: 4DQV will present three papers at CVPR 2022:
  • φ-SfT: Shape-from-Template with a Physics-Based Deformation Model.
  • Physical Inertial Poser (PIP): Physics-aware Real-time Human Motion Tracking from Sparse Inertial Sensors.
  • Playable Environments: Video Manipulation in Space and Time.
• 11.02.2022: State of the art report on neural rendering (the updated version) accepted at EUROGRAPHICS'22.
• 25.10.2021: The source code of Convex Joint Graph Matching and Clustering via Semidefinite Relaxations is released.
• 18.10.2021: An article is accepted at TPAMI (HandVoxNet++). Our method is ranked first on the HANDS19 challenge dataset (Task 1: Depth-Based 3D Hand Pose Estimation) as of August 2020.
• 17.10.2021: Two papers accepted at 3DV 2021 (about graph matching and clustering and human image generation).
• 13.10.2021: EventHands is mentioned in ICCV 2021 Daily.
• 12.10.2021: The source code of GraviCap is released on github.
• 11.10.2021: We have launched the 4DQV group's web page. Check open positions.
• 11.10.2021: The source code of Q-Match is released on github.
• 27.09.2021: The source code of EventHands is released on github.
• 08.09.2021: Dennis Willsch gave a 4DQV Lecture Series talk titled Applications on Quantum Annealers at Jülich Supercomputing Centre. The video can be found here.
• 19.08.2021: We are going to present the following works at ICCV 2021: GraviCap, Non-Rigid Neural Radiance Fields, Iterative Shape Matching via Quantum Annealing and EventHands.
• 05.07.2021: The source code of PhysCap is now available here.
• 03.05.2021: I am starting a research group on 4D and quantum computer vision and accepting now applications for PhD positions, post-doc positions and internships (including theses internships and remote internships).
• 29.04.2020: An article accepted at SIGGRAPH 2021 ("Neural Physcap").
• 19.04.2020: Our work on monocular template-based non-rigid 3D tracking from a single event stream is accepted at CVPR 2021 Workshop on Event-based Vision.
• 30.03.2021: We are going to present three papers at CVPR 2021 (on quantum multi-matching, human motion transfer and human animation from a single image).
• 18.02.2021: Computer Vision and Machine Learning for Computer Graphics, SS 2021, Saarland University.
• 04.02.2021: The source code of PatchNets is now available (github link).
• 30.12.2020: The source code of Non-Rigid Neural Radiance Fields (NR-NeRF) is available (github link).
• 15.10.2020: Three papers accepted to 3DV 2020 (dense NRSfM, quantum graph matching and point set alignment).
• 29.09.2020: Our work on egocentric videoconferencing using a new device with a fisheye camera is accepted at SIGGRAPH Asia 2020.
• 27.08.2020: Our work on physically principled 3D human motion capture in real time is accepted at SIGGRAPH Asia.
• 24.08.2020: Our paper "Neural Re-Rendering of Humans from a Single Image" is featured in "ECCV 2020 Daily".

• PhD positions, post-doc positions and internships are available. Check how to apply.

• Computer Vision and Machine Learning for Computer Graphics, SS 2021, Saarland University.
• If you are interested in a bachelor/master thesis, an internship, research immersion lab (RIL) or a HiWi position on one of the topics listed below, do not hesitate to reach me out.

I am currently leading "4D and Quantum Vision" group at Max Planck Institute for Informatics, D6 Department. The focus of our team lies on 3D reconstruction and analysis of general deformable scenes, 3D reconstruction of the human body and matching problems on point sets and graphs. We are interested in neural approaches (both supervised and unsupervised), physics-based methods as well as new hardware and sensors (e.g., quantum computers and event cameras).

Many research questions at the intersection of computer graphics, computer vision and machine learning involve challenging search problems (e.g., graph matching) or the optimisation of non-convex objectives. For such problems, we develop new algorithmic formulations that can be solved on modern adiabatic quantum annealers or universal quantum computers and investigate which advantages these approaches offer compared to existing classical methods.

Our reserach interests include (but are not limited to):
• 3D Reconstruction and Tracking of Rigid and Non-Rigid Scenes and Objects
• Neural Rendering
• Point Set Registration and Matching Problems
• Quantum Algorithms for Computer Vision and Graphics
• Event-based Approaches in Vision and Graphics

2022

Q-FW: A Hybrid Classical-Quantum Frank-Wolfe for Quadratic Binary Optimization. A. Yurtsever, T, Birdal and V. Golyanik. More details coming soon. [project page] [paper] [bibtex]

UnrealEgo: A New Dataset for Robust Egocentric 3D Human Motion Capture. H. Akada, J. Wang, S. Shimada, M. Takahashi, C. Theobalt and V. Golyanik. European Conference on Computer Vision (ECCV), 2022. [project page] [paper] [bibtex] Description: UnrealEgo is a large-scale naturalistic dataset for egocentric 3D human pose estimation. It is based on an advanced concept of eyeglasses equipped with two fisheye cameras that can be used in unconstrained environments. The experiments show that our simple yet effective approach for egocentric 3D human motion capture outperforms the previous methods.

Quantum Motion Segmentation. F. Arrigoni, W. Menapace, M. Seelbach Benkner, E. Ricci and V. Golyanik. European Conference on Computer Vision (ECCV), 2022. [project page] [paper] [bibtex] Abstract: Motion segmentation is a challenging problem that seeks to identify independent motions in two or several input images. This paper introduces the first algorithm for motion segmentation that relies on adiabatic quantum optimization of the objective function. The proposed method achieves on-par performance with the state of the art on problem instances which can be mapped to modern quantum annealers.

HULC: 3D HUman Motion Capture with Pose Manifold Sampling and Dense Contact Guidance. S. Shimada, V. Golyanik, Z. Li, P. Pérez, W. Xu and C. Theobalt. European Conference on Computer Vision (ECCV), 2022. [paper] [project page]

Neural Radiance Fields for Outdoor Scene Relighting. V. Rudnev, M. Elgharib, W. Smith, L. Liu, V. Golyanik and C. Theobalt. European Conference on Computer Vision (ECCV), 2022. [paper] [project page] [bibtex]

MoCapDeform: Monocular 3D Human Motion Capture in Deformable Scenes. Z. Li, S. Shimada, B. Schiele, C. Theobalt and V. Golyanik. International Conference on 3D Vision (3DV), 2022; Oral. Best Student Paper Award. [project page] [paper] [bibtex] Description: Our MoCapDeform algorithm is the first that models non-rigid scene deformations and finds the accurate global 3D poses of the subject by human-deformable scene interaction constraints, achieving increased accuracy with significantly fewer penetrations.

φ-SfT: Shape-from-Template with a Physics-Based Deformation Model. N. Kairanda, E. Tretschk, M. Elgharib, C. Theobalt and V. Golyanik. Computer Vision and Pattern Recognition (CVPR), 2022. [paper] [project page] [source code] [bibtex]

Physical Inertial Poser (PIP): Physics-aware Real-time Human Motion Tracking from Sparse Inertial Sensors. X. Yi, Y. Zhou, M. Habermann, S. Shimada, V. Golyanik, C. Theobalt and F. Xu. Computer Vision and Pattern Recognition (CVPR), 2022. Best Paper Finalist (one out of 33) [paper] [project page] [source code] [bibtex]

Playable Environments: Video Manipulation in Space and Time. W. Menapace, S. Lathuilière*, A. Siarohin, C. Theobalt*, S. Tulyakov*, V. Golyanik*, and E. Ricci*. * equal senior contribution. Computer Vision and Pattern Recognition (CVPR), 2022. [paper] [project page] [github] [bibtex]

Advances in Neural Rendering. A. Tewari*, J. Thies*, B. Mildenhall*, P. Srinivasan*, E. Tretschk, Y. Wang, C. Lassner, V. Sitzmann, R. Martin-Brualla, S. Lombardi, C. Theobalt, M. Niessner, J. T. Barron, G. Wetzstein, M. Zollhöfer and V. Golyanik. * equal contribution. State of the Art Report at Eurographics 2022. [paper] [project page] [bibtex]

2021

Convex Joint Graph Matching and Clustering via Semidefinite Relaxations. M. Krahn, F. Bernard and V. Golyanik. International Conference on 3D Vision (3DV), 2021. [paper] [project page] [bibtex]

HumanGAN: A Generative Model of Human Images. K. Sarkar, L. Liu, V. Golyanik, and C. Theobalt. International Conference on 3D Vision (3DV), 2021; Oral [paper] [project page] [bibtex]

HandVoxNet++: 3D Hand Shape and Pose Estimation using Voxel-Based Neural Networks. J. Malik, S. Shimada, A. Elhayek, S. A. Ali, C. Theobalt, V. Golyanik and D. Stricker. Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2021. [IEEE Xplore] [arXiv.org] [project page] [bibtex]

Gravity-Aware 3D Human-Object Reconstruction. R. Dabral, S. Shimada, A. Jain, C. Theobalt and V. Golyanik. International Conference on Computer Vision (ICCV), 2021. [paper] [project page] [bibtex]

Q-Match: Iterative Shape Matching via Quantum Annealing. M. Seelbach Benkner, Z. Lähner, V. Golyanik, C. Wunderlich, C. Theobalt and M. Moeller. International Conference on Computer Vision (ICCV), 2021. [paper] [project page] [bibtex]

EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream. V. Rudnev, V. Golyanik, J. Wang, H.-P. Seidel, F. Mueller, M. Elgharib and C. Theobalt. International Conference on Computer Vision (ICCV), 2021. Featured in "ICCV 2021 Daily" (link) [paper] [project page] [source code] [bibtex]

Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video. E. Tretschk, A. Tewari, V. Golyanik, M. Zollhöfer, C. Lassner and C. Theobalt. International Conference on Computer Vision (ICCV), 2021. [paper] [project page] [source code] [bibtex]

Neural Monocular 3D Human Motion Capture with Physical Awareness. ("Neural PhysCap") S. Shimada, V. Golyanik, W. Xu, P. Pérez and C. Theobalt. SIGGRAPH, 2021. [paper] [arXiv] [bibtex] [project page] [source code]

Quantum Permutation Synchronization. T. Birdal*, V. Golyanik*, C. Theobalt and L. Guibas. * equal contribution. Computer Vision and Pattern Recognition (CVPR), 2021. [paper] [arXiv] [project page] [project page (github)] [bibtex]

High-Fidelity Neural Human Motion Transfer from Monocular Video. M. Kappel, V. Golyanik, M. Elgharib, J.-O. Henningson, H.-P. Seidel, S. Castillo, C. Theobalt and M. Magnor. Computer Vision and Pattern Recognition (CVPR), 2021; Oral. [paper] [project page] [bibtex] [source code]

Pose-Guided Human Animation from a Single Image in the Wild. J. S. Yoon, L. Liu, V. Golyanik, K. Sarkar, H. S. Park, and C. Theobalt. Computer Vision and Pattern Recognition (CVPR), 2021. [paper] [project page] [video] [bibtex]

Differentiable Event Stream Simulator for Non-Rigid 3D Tracking. J. Nehvi, V. Golyanik, F. Mueller, H.-P. Seidel, M. Elgharib and C. Theobalt. CVPR Workshop on Event-based Vision, 2021. [paper] [arXiv] [supplement] [source code (upon request)] [bibtex] [project page]

Fast Gravitational Approach for Rigid Point Set Registration with Ordinary Differential Equations. S. A. Ali, K. Kahraman, C. Theobalt, D. Stricker and V. Golyanik. IEEE Access, 2021. [paper] [arXiv] [project page] [bibtex]

2020

PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Time. S. Shimada, V. Golyanik, W. Xu and C. Theobalt. SIGGRAPH Asia, 2020. [paper (arXiv.org)] [bibtex] [project page]

Egocentric Videoconferencing. M. Elgharib*, M. Mendiratta*, J. Thies, M. Nießner, H.-P. Seidel, A. Tewari, V. Golyanik and C. Theobalt. * equal contribution. SIGGRAPH Asia, 2020. [draft] [supplement] [bibtex] [project page]

Fast Simultaneous Gravitational Alignment of Multiple Point Sets. V. Golyanik, S. Shimada and C. Theobalt. 3DV, 2020; Oral. [draft] [bibtex] [project page]

Adiabatic Quantum Graph Matching with Permutation Matrix Constraints. M. Seelbach Benkner, V. Golyanik, C. Theobalt and M. Moeller. 3DV, 2020. [draft] [supplement] [bibtex] [project page]

Intrinsic Dynamic Shape Prior for Dense Non-Rigid Structure from Motion. V. Golyanik, A. Jonas, D. Stricker and C. Theobalt. 3DV, 2020. [draft] [bibtex] [extended version (arXiv)] [3DV talk] [project page]

Neural Dense Non-Rigid Structure from Motion with Latent Space Constraints. V. Sidhu, E. Tretschk, V. Golyanik, A. Agudo and C. Theobalt. European Conference on Computer Vision (ECCV), 2020. [paper] [supplement] [video] [bibtex] [project page (includes the ECCV talks)]

HTML: A Parametric Hand Texture Model for 3D Hand Reconstruction and Personalization. N. Qian, J. Wang, F. Müller, F. Bernard, V. Golyanik and C. Theobalt. European Conference on Computer Vision (ECCV), 2020. [paper] [supplement] [video] [bibtex] [project page]

PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations. E. Tretschk, A. Tewari, V. Golyanik, M. Zollhöfer, C. Stoll and C. Theobalt. European Conference on Computer Vision (ECCV), 2020. [paper] [supplement] [video] [bibtex] [project page (includes the ECCV talks)] [source code]

DEMEA: Deep Mesh Autoencoders for Non-Rigidly Deforming Objects. E. Tretschk, A. Tewari, M. Zollhöfer, V. Golyanik and C. Theobalt. European Conference on Computer Vision (ECCV), 2020; Spotlight [paper] [supplement] [bibtex] [arXiv] [project page (includes the ECCV talks)]

Neural Re-Rendering of Humans from a Single Image. K. Sarkar, D. Mehta, W. Xu, V. Golyanik and C. Theobalt. European Conference on Computer Vision (ECCV), 2020. Featured in "ECCV 2020 Daily" (link) [paper] [supplement] [video] [bibtex] [project page]

A Quantum Computational Approach to Correspondence Problems on Point Sets. V. Golyanik and C. Theobalt. In Computer Vision and Pattern Recognition (CVPR), 2020. [paper] [slides] [poster] [bibtex] [arXiv] [project page]

EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Camera. L. Xu, W. Xu, V. Golyanik, M. Habermann, L. Fang and C. Theobalt. In Computer Vision and Pattern Recognition (CVPR), 2020; Oral [paper] [supplement] [bibtex] [arXiv] [project page]

HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Map. J. Malik, I. Abdelaziz, A. Elhayek, S. Shimada, S. A. Ali, V. Golyanik, C. Theobalt and D. Stricker. In Computer Vision and Pattern Recognition (CVPR), 2020. [paper] [supplement] [bibtex] [arXiv] [project page]

Robust Methods for Dense Monocular Non-Rigid 3D Reconstruction and Alignment of Point Clouds. Doctoral Dissertation, TU Kaiserslautern. V. Golyanik. Springer Vieweg Verlag, 2020. DAGM MVTec Dissertation Award 2020 [Springer] [Springer Link] [Book Flyer]

2019

Structure from Articulated Motion: Accurate and Stable Monocular 3D Reconstruction without Training Data. O. Kovalenko, V. Golyanik, J. Malik, A. Elhayek and D. Stricker. Sensors (Volume 19, Issue 20), 2019. [paper] [project page]

Accelerated Gravitational Point Set Alignment with Altered Physical Laws. V. Golyanik, C. Theobalt and D. Stricker. International Conference on Computer Vision (ICCV), 2019; [paper] [supplementary material] [poster] [bibtex] [project page]

A Shape Completion Component for Monocular Non-Rigid SLAM. Y. Su, V. Golyanik, N. Minaskan, S. A. Ali and D. Stricker. International Symposium on Mixed and Augmented Reality (ISMAR), 2019. [paper] [supplement (video, 15 MB)] [MSCC Dataset] [bibtex]

DispVoxNets: Non-Rigid Point Set Alignment with Supervised Learning Proxies. S. Shimada, V. Golyanik, E. Tretschk, D. Stricker and C. Theobalt. In International Conference on 3D Vision (3DV), 2019; Oral [paper] [poster] [presentation] [project page] [arXiv] [bibtex]

Optimising for Scale in Globally Multiply-Linked Gravitational Point Set Registration Leads to Singularities. V. Golyanik and C. Theobalt. In International Conference on 3D Vision (3DV), 2019; Spotlight [paper] [supplement (pdf)] [poster] [video] [bibtex]

FACE IT!: A Pipeline For Real-Time Performance-Driven Facial Animation. J. M. Dı́az Barros, V. Golyanik, K. Varanasi and D. Stricker. International Conference on Image Processing ICIP, 2019; Oral (Lecture) [paper] [bibtex]

IsMo-GAN: Adversarial Learning for Monocular Non-Rigid 3D Reconstruction. S. Shimada, V. Golyanik, C. Theobalt and D. Stricker. Computer Vision and Pattern Recognition Workshops (Photogrammetric Computer Vision Workshop), 2019; Oral [paper] [bibtex] [arXiv] [project page]

Consolidating Segmentwise Non-Rigid Structure from Motion. V. Golyanik, A. Jonas and D. Stricker. Machine Vision Applications (MVA), 2019; Oral [paper] [project page] [bibtex]

2018

NRGA: Gravitational Approach for Non-Rigid Point Set Registration. S. A. Ali. V. Golyanik and D. Stricker. International Conference on 3D Vision (3DV), 2018; Oral [paper] [Supplementary Video (Download, YouTube)] [poster] [bibtex]

HDM-Net: Monocular Non-Rigid 3D Reconstruction with Learned Deformation Model. V. Golyanik, S. Shimada, K. Varanasi and D. Stricker. EuroVR, 2018; Oral (Long Paper) [paper] [HDM-Net data set] [bibtex]

Improving Time-Of-Flight Sensor for Specular Surfaces With Shape from Polarozation. T. Yoshida, V. Golyanik, O. Wasenmüller and D. Stricker. International Conference on Image Processing (ICIP), 2018. [paper] [bibtex]

Classification of LIDAR Sensor Contaminations with Deep Neural Networks. J. K. James, G. Puhlfürst, V. Golyanik and D. Stricker. ACM Chapters Computer Science in Cars Symposium (CSCS), 2018; Oral [paper] [bibtex]

2017

Multiframe Scene Flow with Piecewise Rigid Motion. V. Golyanik, K. Kim, R. Maier, M. Nießner, D. Stricker and J. Kautz. International Conference on 3D Vision (3DV), 2017; Spotlight Oral [paper] [arXiv] [supplementary material] [poster] [bibtex]

Scalable Dense Monocular Surface Reconstruction. M.D.Ansari, V. Golyanik and D. Stricker. International Conference on 3D Vision (3DV), 2017. [paper] [arXiv] [bibtex]

High-Dimensional Model for Dense Monocular Surface Recovery. V. Golyanik and D. Stricker. International Conference on 3D Vision (3DV), 2017. [paper] [bibtex]

Introduction to Coherent Depth Fields for Dense Monocular Surface Recovery. V. Golyanik, T. Fetzer and D. Stricker. British Machine Vision Conference (BMVC), 2017. [paper] [supplementary video] [bibtex]

Towards Scheduling Hard Real-Time Image Processing Tasks on a Single GPU. V. Golyanik, M. Nasri and D. Stricker. International Conference on Image Processing (ICIP), 2017. [paper] [bibtex]

A Framework for an Accurate Point Cloud Based Registration of Full 3D Human Body Scans. V. Golyanik, G. Reis, B. Taetz and D. Stricker. Machine Vision Applications (MVA), 2017. [paper] [bibtex]

Dense Batch Non-Rigid Structure from Motion in a Second. V. Golyanik and D. Stricker. Winter Conference on Applications of Computer Vision (WACV), 2017. [paper] [supplementary video] [poster] [bibtex]

Accurate 3D Reconstruction of Dynamic Scenes from Monocular Image Sequences with Severe Occlusions. V. Golyanik, T. Fetzer and D. Stricker. Winter Conference on Applications of Computer Vision (WACV), 2017. [paper] [supplementary material] [poster] [arXiv] [bibtex]

2016

NRSfM-Flow: Recovering Non-Rigid Scene Flow from Monocular Image Sequences. V. Golyanik, A. S. Mathur and D. Stricker. British Machine Vision Conference (BMVC), 2016. [paper] [extended abstract] [poster] [supplementary video] [bibtex]

Joint Pre-Alignment and Robust Rigid Point Set Registration. V. Golyanik, B. Taetz and D. Stricker. International Conference on Image Processing (ICIP), 2016. [paper] [bibtex]

Gravitational Approach for Point Set Registration. V. Golyanik, S. A. Ali and D. Stricker. Computer Vision and Pattern Recognition (CVPR), 2016. [paper] [supplementary material] [bibtex]

Extended Coherent Point Drift Algorithm with Correspondence Priors and Optimal Subsampling. V. Golyanik, B. Taetz, G. Reis and D. Stricker. Winter Conference on Applications of Computer Vision (WACV), 2016. [paper] [poster] [bibtex] [WACV Talk]

Occlusion-Aware Video Registration for Highly Non-Rigid Objects. B. Taetz, G. Bleser, V. Golyanik and D. Stricker. Winter Conference on Applications of Computer Vision (WACV), 2016. Best Paper Award. [paper] [supplementary material] [bibtex] [WACV Talk]

2015

Precise and Automatic Anthropometric Measurement Extraction using Template Registration. O. Wasenmüller, J. C. Peters, V. Golyanik and D. Stricker. International Conference on 3D Body Scanning Technologies (3DBST), 2015. [paper] [bibtex]

• Since Sept.2018:
  Postdoctoral Researcher at Max Planck Institute (MPI) for Informatics
• Jan.2016 - Sept.2018:
  Researcher at German Center for Artificial Intelligence (DFKI),
  Augmented Vision Department, Kaiserslautern, Germany
• Oct.2016 - April.2017:
  Visiting Research Intern at NVIDIA, Santa Clara, CA, USA
• March.2014 - Dec.2015:
  Researcher at the University of Kaiserslautern, Department of Computer Science,
  Augmented Vision Lab (operated in cooperation with DFKI)