Firstname Lastname

Soshi Shimada

Max-Planck-Institut für Informatik
D4: Computer Graphics
 office: Campus E1 4, Room #room#
Saarland Informatics Campus
66123 Saarbrücken
 email: Get my email address via email
 phone: +49 681 9325-4055
 fax: +49 681 9325-4099

Research Interests

  • Machine Learning
  • 3D Reconstruction
  • Non-Rigid Point Set Registration



    HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Map
    J. Malik, I. Abdelaziz, A. Elhayek, S. Shimada, S. A. Ali, V. Golyanik, C. Theobalt and D. Stricker.
    Accepted in Computer Vision and Pattern Recognition (CVPR), 2020.

    we propose a novel architecture with 3D convolutions trained in a weakly-supervised manner. The input to our method is a 3D voxelized depth map, and we rely on two hand shape representations. The first one is the 3D voxelized grid of the shape which is accurate but does not preserve the mesh topology and the number of mesh vertices. The second representation is the 3D hand surface which is less accurate but does not suffer from the limitations of the first representation. We combine the advantages of these two representations by registering the hand surface to the voxelized hand shape.
    [paper] [supplement] [project page] [arXiv]

    DispVoxNets: Non-Rigid Point Set Alignment with Supervised Learning Proxies.
    S. Shimada, V. Golyanik, E. Tretschk, D. Stricker and C. Theobalt.
    In International Conference on 3D Vision (3DV), 2019; Oral

    We introduce a supervised-learning framework for nonrigid point set alignment of a new kind — Displacements on Voxels Networks (DispVoxNets) — which abstracts away from the point set representation and regresses 3D displacement fields on regularly sampled proxy 3D voxel grids. Thanks to recently released collections of deformable objects with known intra-state correspondences, DispVoxNets learn a deformation model and further priors (e.g., weak point topology preservation) for different object categoriessuch as cloths, human bodies and faces.
    [paper] [poster] [project page] [arXiv]

    IsMo-GAN: Adversarial Learning for Monocular Non-Rigid 3D Reconstruction.
    S. Shimada, V. Golyanik, C. Theobalt and D. Stricker.
    In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2019; Oral

    The majority of the existing methods for non-rigid 3D surface regression from a single 2D image require an object template or point tracks over multiple frames as an input, and are still far from real-time processing rates. In this work, we present the Isometry-Aware Monocular Generative Adversarial Network (IsMo-GAN) — an approach for direct 3D reconstruction from a single image, trained for the deformation model in an adversarial manner on a light-weight synthetic dataset.
    [paper] [arXiv]
    HDM-Net: Monocular Non-Rigid 3D Reconstruction with Learned Deformation Model.
    V. Golyanik, S. Shimada, K. Varanasi and D. Stricker.
    In International Conference on Virtual Reality and Augmented Reality (EuroVR) 2018; Oral (Long Paper)

    Monocular dense 3D reconstruction of deformable objects is a hard ill-posed problem in computer vision.In this work, we propose a new hybrid approach for monocular non-rigid reconstruction which we call Hybrid Deformation Model Network (HDM-Net). In our approach, a deformation model is learned by a deep neural network, with a combination of domainspecific loss functions.
    [paper] [HDM-Net data set]

Recent Positions