MPI-INF Logo
Homepage

Contact

Firstname Lastname

Mohamed Elgharib

Research Group Leader
Vision-based Graphics

Max-Planck-Institut für Informatik
D6: Visual Computing and Artificial Intelligence
 office: Campus E1 4, Room 218
Saarland Informatics Campus
66123 Saarbrücken
Germany
 email: elgharib [at] mpi-inf [dot] mpg [dot] de
 phone: +49 681 9325-4506
 fax: +49 681 9325-4099

Research Interests

I am a Research Group Leader at the Max Planck Institute for Informatics. We design methods for digitizing our surroundings. To this end, we use data captured by monocular RGB cameras and other sensors to model, reconstruct and videorealistically render our world. Human faces are at the forefront of our visual communications, and hence their processing is of a special interest to us, with applications in visual telepresence in our mind. Our work lies in the intersection of computer vision and computer graphics, and usually includes a heavy machine and deep learning component through supervised, self-supervised or unsupervised learning. Our end goal is to fully digitize our world to allow all of us to connect and interact with our friends, family, and loved ones, over a distance.

Topics of interest include:  
  • 3D scene modeling and reconstruction
  • Deep generative models
  • Videorealistic neural rendering
  • Digital faces
  • Pose estimation
  • Relighting
  • Event camreas

We are hiring

If you are interested in Master's thesis, Research Immersion Lab (RIL) or Hiwi positions, feel free to contact us. We are also always looking for motivated PhD students.

Selected Publications

    AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars
    M. Mendiratta, X. Pan*, M. Elgharib*, K. Teotia, Mallikarjun B R, A. Tewari, V. Golyanik, A. Kortylewski and C. Theobalt (* equal contribution)
    To appear in ACM Transactions on Graphics (Proceedings of SIGGRAPH - Asia 2023)
    [Manuscript], [Project Page]

    A Deeper Analysis of Volumetric Relightable Faces
    P. Rao, M. B R, G. Fox, T. Weyrich, B. Bickel, H. Pfister, W. Matusik, F. Zhan, A. Tewari, C. Theobalt and M. Elgharib
    International Journal of Computer Vision 2023 (IJCV - Invited Paper)
    [Manuscript]

    LiveHand: Real-time and Photorealistic Neural Hand Rendering
    A. Mundra, M. B R, J. Wang, M. Habermann, C. Theobalt and M. Elgharib
    To appear in the International Conference on Computer Vision 2023 (ICCV)
    [Manuscript], [Project Page]

    EventNeRF: Neural Radiance Fields from a Single Colour Event Camera
    V. Rudnev, M. Elgharib, C. Theobalt and V. Golyanik
    In Computer Vision and Pattern Recognition 2023 (CVPR)
    [Manuscript], [Project Page]

    An Implicit Parametric Morphable Dental Model
    C. Zhang, M. Elgharib, G. Fox, M. Gu, C. Theobalt and W. Wang
    In ACM Transactions on Graphics (Proceedings of SIGGRAPH - Asia 2022)
    [Manuscript], [Project Page]

    VoRF: Volumetric Relightable Faces
    P. Rao, M. B R, G. Fox, T. Weyrich, B. Bickel, H. Pfister, W. Matusik, A. Tewari, C. Theobalt and M. Elgharib
    To appear in the British Machine Vision Conference 2022 (BMVC) - Oral - Best Paper Award Honourable Mention
    [Manuscript], [Project Page]

    gCoRF: Generative Compositional Radiance Fields
    M. B R, A. Tewari, X. Pan, M. Elgharib and C. Theobalt
    In the International Conference on 3D Vision 2022 (3DV) - Spotlight Presentation
    [Manuscript], [Project Page]

    Neural Radiance Fields for Outdoor Scene Relighting
    V. Rudnev, M. Elgharib, W. Smith, L. Liu, V. Golyanik and C. Theobalt
    European Conference on Computer Vision 2022 (ECCV)
    [Manuscript], [Project Page]

    A Motion Matching-based Framework for Controllable Gesture Synthesis from Speech
    I. Habibie, M. Elgharib, K. Sarkar, A. Abdullah, S., M. Neff and C. Theobalt
    SIGGRAPH Conference 2022
    [Manuscript], [Project Page]

    φ-SfT: Shape-from-Template with a Physics-Based Deformation Model
    N. Kairanda, E. Tretschk, M. Elgharib, C. Theobalt and V. Golyanik.
    Computer Vision and Pattern Recognition 2022 (CVPR)
    [Manuscript], [Project Page]

    StyleVideoGAN2: A Temporal Generative Model using a Pretrained StyleGAN
    G. Fox, A. Tewari, M. Elgharib and C. Theobalt
    British Machine Vision Conference 2021(BMVC) - Oral
    [Manuscript], [Project Page]

    EventHands: Real-Time Neural 3D Hand Reconstruction from an Event Stream
    V. Rudnev, V. Golyanik, J. Wang, H-P. Seidel, F. Mueller, M. Elgharib and C. Theobalt
    International Conference on Computer Vision 2021 (ICCV)
    [Manuscript], [video], [Project Page], [ICCV Daily]

    Learning Speech-driven 3D Conversational Gestures from Video
    I. Habibie, W. Xu, D. Mehta, L. Liu, H-P. Seidel, G. Pons-Moll, M. Elgharib and C. Theobalt
    ACM International Conference on Intelligent Virtual Agents 2021 (IVA) - Best Paper Award
    [Manuscript], [Project Page]

    PhotoApp: Photorealistic Appearance Editing of Head Portraits
    M. B R, A. Tewari, A. Dib, T. Weyrich, B. Bickel, H-P. Seidel, H. Pfister, W. Matusik, L. Chevallier, M. Elgharib and C. Theobalt
    ACM Transactions on Graphics (Proceedings of SIGGRAPH - 2021)
    [Manuscript], [Project Page]

    High-Fidelity Neural Human Motion Transfer from Monocular Video
    M. Kappel, V. Golyanik, M. Elgharib, J.-O. Henningson, H-P. Seidel, S. Castillo, C. Theobalt and M. Magnor
    Computer Vision and Pattern Recognition 2021 (CVPR) - Oral
    [Manuscript], [video], [Project Page]

    i3DMM: Deep Implicit 3D Morphable Model of Human Heads
    T. Yenamandra, A. Tewari, F. Bernard, H-P. Seidel, M. Elgharib, D. Cremers and C. Theobalt
    Computer Vision and Pattern Recognition 2021 (CVPR) - Oral
    [Manuscript], [video], [Project Page]

    Monocular Reconstruction of Neural Face Reflectance Fields
    M. B R, A. Tewari, T-H. Oh, T. Weyrich, B. Bickel, H-P. Seidel, H. Pfister, W. Matusik, M. Elgharib and C. Theobalt
    Computer Vision and Pattern Recognition 2021 (CVPR)
    [Manuscript], [Project Page]

    Learning Complete 3D Morphable Face Models from Images and Videos
    M. B R, A. Tewari, H-P. Seidel, M. Elgharib and C. Theobalt
    Computer Vision and Pattern Recognition 2021 (CVPR)
    [Manuscript], [Project Page]

    VideoForensicsHQ: Detecting High-quality Manipulated Face Videos
    G. Fox, W. Liu, H. Kim, H-P. Seidel, M. Elgharib and C. Theobalt
    IEEE International Conference on Multimedia and Expo 2021 (ICME) - Oral
    [Manuscript], [Project Page]

    Egocentric Videoconferencing
    M. Elgharib*, M. Mendiratta*, J. Thies, M. Niessner, H-P. Seidel, A. Tewari, V. Golyanik and C. Theobalt (* equal contribution)
    ACM Transactions on Graphics (Proceedings of SIGGRAPH - Asia 2020)
    [Manuscript], [Project Page], [Video], [Two Minute Papers!]

    PIE: Portrait Image Embedding for Semantic Control
    A. Tewari, M. Elgharib, M. B R, F. Bernard, H-P. Seidel, P. Perez, M. Zollhöfer and C. Theobalt
    ACM Transactions on Graphics (Proceedings of SIGGRAPH - Asia 2020)
    [Manuscript], [Project Page]

    Neural Voice Puppetry: Audio-driven Facial Reenactment
    J. Thies, M. Elgharib, A. Tewari, C. Theobalt and M. Niessner
    European Conference on Computer Vision 2020 (ECCV)
    [Manuscript], [Project page], [Demo], [Source code]

    Self-supervised Outdoor Scene Relighting
    Y. Yu, A. Meka, M. Elgharib, H.P. Seidel, C. Theobalt and W. Smith
    European Conference on Computer Vision 2020 (ECCV)
    [Manuscript], [Project page]

    XNect: Real-time Multi-person 3D Motion Capture with a Single RGB Camera
    D. Mehta, O. Sotnychenko, F. Mueller, W. Xu, M. Elgharib, P. Fua, H.P. Seidel, H. Rhodin, G. Pons-Moll and C. Theobalt
    ACM Transactions on Graphics (Proceedings of SIGGRAPH 2020)
    [Manuscript], [Project page], [Demo@CVPR19], [SelecSLS-Pytorch]

    StyleRig: Rigging StyleGAN for 3D Control over Portrait Images
    A. Tewari, M. Elgharib, G. Bharaj, F. Bernard, H-P. Seidel, P. Perez, M. Zollhöfer and C. Theobalt
    Computer Vision and Pattern Recognition 2020 (CVPR) - Oral
    [Manuscript], [Project page]

    Neural Style-Preserving Visual Dubbing
    H. Kim, M. Elgharib, M. Zollhofer, T. Beeler, H.P. Seidel, C. Richardt and C. Theobalt.
    ACM Transactions on Graphics (Proceedings of SIGGRAPH - Asia 2019)
    [Manuscript], [Project page]

    FML: Face Model Learning
    A. Tewari, F. Bernard, P. Garrido, G. Bharaj, M. Elgharib, H.P. Seidel, P. Perez, M. Zollhofer and C. Theobalt.
    Computer Vision and Pattern Recognition 2019 (CVPR) - Oral
    [Manuscript], [Project page]

    On Learning Associations of Faces and Voices
    C. Kim, H. V. Shin, T. H. Oh, A. Kaspar, M. Elgharib, and W. Matusik.
    Asian Conference on Computer Vision 2018 (ACCV)
    [Manuscript], [Project page]

    Learning-based Video Motion Magnification
    T. H. Oh*, R. Jaroensri*, C. Kim, M. Elgharib, F. Durand, W. T. Freeman, and W. Matusik. *indicates equal contributions
    European Conference on Computer Vision 2018 (ECCV) - Oral
    [Manuscript], [Project page]

    A Dataset of Flash and Ambient Illumination Pairs from the Crowd
    Y. Aksoy, C. Kim, P. Kellnhofer, S. Paris, M. Elgharib, M. Pollefeys, and W. Matusik.
    European Conference on Computer Vision 2018 (ECCV)
    [Manuscript], [Project page]

    Crowd-Guided Ensembles. How Can We Choreograph Crowd Workers for Video Segmentation?
    A. Kasper, G. Patterson, C. Kim, Y. Aksoy, W. Matusik, and M. Elgharib.
    ACM Conference on Human Factors in Computing Systems 2018 (CHI) - Oral
    [Manuscript], [Video teaser], [Project page]

    Video reflection removal through spatio-temporal optimization
    A. Nandoriya*, M. Elgharib*, C. Kim, M. Hefeeda, and W. Matusik. *indicates equal contrbutions.
    International Conference on Computer Vision 2017 (ICCV)
    [Manuscript], [Project page]

    Sports VR Content Generation from Regular Camera Feeds
    K. Calagari, M. Elgharib, S. Shirmohammadi, and M. Hefeeda.
    ACM Multimedia 2017 (ACM MM) - Full Paper
    [Manuscript], [Project page]

    Data Driven 2-D to 3-D Video Conversion for Soccer
    K. Calagari, M. Elgharib, P. Didyk, A. Kaspar, W. Matusik, and M. Hefeeda.
    IEEE Transactions on Multimedia 2017 (TMM)
    [Manuscript], [Project page]

    Painting Style Transfer for Head Portraits using Convolutional Neural Networks
    A. Selim*, M. Elgharib*, and L. Doyle. *joint first author, sorted alphabetically
    ACM Transactions on Graphics (Proceedings of SIGGRAPH 2016)
    Selected as the proceedings back-cover!
    [Manuscript], [Video teaser], [Project page]

    Gradient-based 2-D to 3-D conversion for Soccer Videos
    K. Calagari,M. Elgharib, P. Didyk, A. Kaspar, W. Matusik, and M. Hefeeda.
    ACM Multimedia 2015 (ACM MM) - Oral
    Selected as MIT home-page!
    [Manuscript], [MIT main page], [Project page]

    Video Magnification in Presence of Large Motions
    M. Elgharib, M. Hefeeda, F. Durand and W. T. Freeman.
    Computer Vision and Pattern Recongition 2015 (CVPR)
    [Manuscript], [Project page]

    Retrieval in Long Surveillance Videos using User Described Motion and Object Attributes
    G. Castanon, M. Elgharib, P.M. Jodoin and V. Saligrama.
    IEEE Transactions on Circuits and Systems for Video Technology, 26:12, Dec. 2016 (accepted Aug. 2015) (TCSVT)
    [Manuscript], [Project page]

    Blotch and scratch removal in archived film using a semi-transparent corruption model and a ground-truth generation technique
    M. Elgharib, F. Pitie, and Anil Kokaram.
    EURASIP Journal on Image and Video Processing 2013:33 (EURASIP)
    [Manuscript], [Project page]

    Motion Estimation for Regions of Reflections through Layer Separation
    M. Elgharib, F. Pitie, and Anil Kokaram.
    ACM SIGGRAPH European Conference on Visual Media Production 2011 (CVMP) - Oral
    [Manuscript], [Project page]

    Reflection Detection in Image Sequences
    M. Elgharib, F. Pitie, and Anil Kokaram.
    Computer Vision and Pattern Recognition 2011 (CVPR)
    [Manuscript], [Project page]

    Extraction of Non-binary Blotch Mattes
    M. Elgharib, F. Pitie, and Anil Kokaram.
    International Conference on Image Processing 2009 (ICIP)
    [Manuscript], [Project page]

Technical Reports

  • A. Hassanein, M. Elgharib, A. Selim, S. Bae, M. Hefeeda and W. Matusik, Large-scale, Fast and Accurate Shot Boundary Detection through Spatio-temporal Convolutional Neural Networks, 2017. [Project page] [Dataset] [Code and models]
  • M. Elgharib, Z. Wu, R. Radke and V. Saligarama, Distributed Video Analytics and Anomaly Detection, 2014.
  • Teaching

  • C. Theobalt, M. Elgharib, V. Golyanik, Computer Vision and Machine Learning for Computer Graphics, University of Saarland, 2021.
  • C. Theobalt, M. Elgharib, V. Golyanik, Computer Vision and Machine Learning for Computer Graphics, University of Saarland, 2020.
  • C. Theobalt, M. Elgharib, V. Golyanik, Computer Vision and Machine Learning for Computer Graphics, University of Saarland, 2019.
  • Granted U.S. Patents

  • M. Elgharib, W. Matusik, S.-H. Bae and M. Hefeeda, View generation from a single image using fully convolutional neural networks, 2018.
  • M. Elgharib, M. Hefeeda, W. Freeman and F. Durand, Method for dynamic video magnification, 2015.
  • M. Hefeeda, K. Calagari, M. Elgharib, W. Matusik, P. Didyk and A. Kaspar, 2D-to-3D video frame conversion, 2015.
  • Previous Positions

  • November 2016 - July 2018: Scientist, Qatar Computing Research Institute (QCRI).
    • I was co-leading a collaborative QCRI-MIT project. On average, the team included 3 research assistants and 1 post-doc, and produced 5 publications [1,2,3,4,5], 1 well-cited technical report [6] and 1 US granted patent [7].
  • October 2013 - November 2016: Post-doc, Qatar Computing Research Institute (QCRI).
  • January 2012 - October 2013: Boston Unversity.
    • Developed reliable and computationally efficient video search techniques for surveillance footage captured in harsh conditions.
  • Education

  • 2011: PhD in Electrical and Computer Engineering, at Trinity College Dublin, Ireland - Full paid non-EU scholarship from IRCSET.
  • 2008: B.A.I in Electrical and Computer Engineering, at Trinity College Dublin, Ireland - Gold Medalist.
  • MISC

  • Reviewer for CVPR, ICCV, ECCV, SIGGAPH, SIGGRAPH-Asia, NeurIPS, PAMI, TIPS, 3DV, ACCV, and others.
  • Hobbies: I enjoy travelling, sightseeing, walking, hiking, natural scenery, watching documentaries and catching up with friends!