MPI-INF Logo
Homepage

Contact

Firstname Lastname

Mohamed Elgharib

Max-Planck-Institut für Informatik
D6: Visual Computing and Artificial Intelligence
 office: Campus E1 4, Room 218
Saarland Informatics Campus
66123 Saarbrücken
Germany
 email: elgharib [at] mpi-inf [dot] mpg [dot] de
 phone: +49 681 9325-4018
 fax: +49 681 9325-4099

NEWS

  • NEW: 01.05.21: Appointed as a research group leader at D6. I will be establishing the “Vision-based Graphics“ (VbG) group. The group will work on methods for digitizing and controlling our surroundings in a videorealistic manner. We are hiring at all levels (Postdocs, PhDs, Masters, RIL, Hiwi's, Interns, etc..). Don't hestiate to contact me if interested!
  • 20.04.21: Our work on StyleGAN-based facial relighting is accepted for publication in SIGGRAPH '21.
  • 15.04.21: Our paper on event camera based non-rigid 3D tracking is accepted for publcation.
  • 13.04.21: For more information on our SS'21 seminar please check out this website.
  • 10.03.21: Our work on automatically detecting highly videorealistic portraits is accepted for publication in ICME '21 as an oral presentation.
  • 05.03.21: 4 papers accepted in CVPR '21, including 2 Orals [1,2,3,4].
  • 17.11.20: Our paper on egocentric videoconferencing got featured in Two Minute Papers!
  • 29.10.20: Two papers [1,2] accepted for publication in SIGGRAPH (Asia) '20.
  • 01.08.20: Two papers [1,2] accepted for publication in ECCV '20.
  • 01.06.20: XNect is accepted for publication in SIGGRAPH '20.
  • 15.03.20: Our work on rigging StyleGAN images is accepted for publication in CVPR '20.
  • 01.09.19: OurX work on style-preserving visual dubbing is accepted for publication in SIGGRAPH (Asia) '19.

We are hiring!

  • NEW: We have openings at all levels (Postdocs, PhDs, Masters, Interns, etc..). We offer topics for Master thesis and Research Immersion Lab. We are also open for collaborations with the possibility of spending time at MPII. Please contact me if interested.

Research Interests

I lead the "Vision-based Graphics" (VbG) group. In the VbG group we design methods for digitizing and controlling our surroundings in a videorealistic manner. We use data captured by monocular RGB cameras, and sometimes by other sensors, to reconstruct and render models of the real world. Human faces are at the forefront of our visual communications, and hence their processing is of a special interest to us, with applications in visual telepresence in our mind. Our work lies in the intersection of computer vision and computer graphics, and tackles problems through data-driven and neural based formulations. Our end goal is to fully digitize our surroundings to allow all of us to connect and interact with our friends, family, and loved ones, over a distance.

Topics of interest include:  
  • Digital Faces
  • Videorealistic rendering
  • 3D scene modeling and reconstruction
  • Deep generative models
  • Motion handling through Event and RGB cameras

Publications

    EventHands: Real-Time Neural 3D Hand Reconstruction from an Event Stream
    V. Rudnev, V. Golyanik, J. Wang, H-P. Seidel, F. Mueller, M. Elgharib and C. Theobalt
    To appear in the International Conference on Computer Vision 2021 (ICCV)
    [Manuscript], [video], [Project Page]

    Learning Speech-driven 3D Conversational Gestures from Video
    I. Habibie, W. Xu, D. Mehta, L. Liu, H-P. Seidel, G. Pons-Moll, M. Elgharib and C. Theobalt
    To appear in ACM International Conference on Intelligent Virtual Agents 2021 (IVA)
    [Manuscript], [Project Page (coming soon)]

    PhotoApp: Photorealistic Appearance Editing of Head Portraits
    M. B R, A. Tewari, A. Dib, T. Weyrich, B. Bickel, H-P. Seidel, H. Pfister, W. Matusik, L. Chevallier, M. Elgharib and C. Theobalt
    To appear in ACM Transactions on Graphics (Proceedings of SIGGRAPH - 2021)
    [Manuscript], [Project Page]

    High-Fidelity Neural Human Motion Transfer from Monocular Video
    M. Kappel, V. Golyanik, M. Elgharib, J.-O. Henningson, H-P. Seidel, S. Castillo, C. Theobalt and M. Magnor
    Computer Vision and Pattern Recognition 2021 (CVPR) - Oral
    [Manuscript], [video], [Project Page]

    i3DMM: Deep Implicit 3D Morphable Model of Human Heads
    T. Yenamandra, A. Tewari, F. Bernard, H-P. Seidel, M. Elgharib, D. Cremers and C. Theobalt
    Computer Vision and Pattern Recognition 2021 (CVPR) - Oral
    [Manuscript], [video], [Project Page]

    Monocular Reconstruction of Neural Face Reflectance Fields
    M. B R, A. Tewari, T-H. Oh, T. Weyrich, B. Bickel, H-P. Seidel, H. Pfister, W. Matusik, M. Elgharib and C. Theobalt
    Computer Vision and Pattern Recognition 2021 (CVPR)
    [Manuscript], [Project Page]

    VideoForensicsHQ: Detecting High-quality Manipulated Face Videos
    G. Fox, W. Liu, H. Kim, H-P. Seidel, M. Elgharib and C. Theobalt
    IEEE International Conference on Multimedia and Expo 2021 (ICME) - Oral
    [Manuscript], [Project Page]

    Learning Complete 3D Morphable Face Models from Images and Videos
    M. B R, A. Tewari, H-P. Seidel, M. Elgharib and C. Theobalt
    Computer Vision and Pattern Recognition 2021 (CVPR)
    [Manuscript], [Project Page]

    Egocentric Videoconferencing
    M. Elgharib*, M. Mendiratta*, J. Thies, M. Niessner, H-P. Seidel, A. Tewari, V. Golyanik and C. Theobalt (* equal contribution)
    ACM Transactions on Graphics (Proceedings of SIGGRAPH - Asia 2020)
    [Manuscript], [Project Page], [Video], [Two Minute Papers!]

    PIE: Portrait Image Embedding for Semantic Control
    A. Tewari, M. Elgharib, M. B R, F. Bernard, H-P. Seidel, P. Perez, M. Zollhöfer and C. Theobalt
    ACM Transactions on Graphics (Proceedings of SIGGRAPH - Asia 2020)
    [Manuscript], [Project Page]

    Neural Voice Puppetry: Audio-driven Facial Reenactment
    J. Thies, M. Elgharib, A. Tewari, C. Theobalt and M. Niessner
    European Conference on Computer Vision (ECCV 2020)
    [Manuscript], [Project page], [Demo], [Source code]

    Self-supervised Outdoor Scene Relighting
    Y. Yu, A. Meka, M. Elgharib, H.P. Seidel, C. Theobalt and W. Smith
    European Conference on Computer Vision (ECCV 2020)
    [Manuscript], [Project page]

    XNect: Real-time Multi-person 3D Motion Capture with a Single RGB Camera
    D. Mehta, O. Sotnychenko, F. Mueller, W. Xu, M. Elgharib, P. Fua, H.P. Seidel, H. Rhodin, G. Pons-Moll and C. Theobalt
    ACM Transactions on Graphics (Proceedings of SIGGRAPH 2020)
    [Manuscript], [Project page], [Demo@CVPR19], [SelecSLS-Pytorch]

    StyleRig: Rigging StyleGAN for 3D Control over Portrait Images
    A. Tewari, M. Elgharib, G. Bharaj, F. Bernard, H-P. Seidel, P. Perez, M. Zollhöfer and C. Theobalt
    Computer Vision and Pattern Recognition 2020 (CVPR) - Oral
    [Manuscript], [Project page]

    Neural Style-Preserving Visual Dubbing
    H. Kim, M. Elgharib, M. Zollhofer, T. Beeler, H.P. Seidel, C. Richardt and C. Theobalt.
    ACM Transactions on Graphics (Proceedings of SIGGRAPH - Asia 2019)
    [Manuscript], [Project page]

    FML: Face Model Learning
    A. Tewari, F. Bernard, P. Garrido, G. Bharaj, M. Elgharib, H.P. Seidel, P. Perez, M. Zollhofer and C. Theobalt.
    Computer Vision and Pattern Recognition 2019 (CVPR) - Oral
    [Manuscript], [Project page]

    On Learning Associations of Faces and Voices
    C. Kim, H. V. Shin, T. H. Oh, A. Kaspar, M. Elgharib, and W. Matusik.
    Asian Conference on Computer Vision 2018 (ACCV)
    [Manuscript], [Project page]

    Learning-based Video Motion Magnification
    T. H. Oh*, R. Jaroensri*, C. Kim, M. Elgharib, F. Durand, W. T. Freeman, and W. Matusik. *indicates equal contributions
    European Conference on Computer Vision 2018 (ECCV) - Oral
    [Manuscript], [Project page]

    A Dataset of Flash and Ambient Illumination Pairs from the Crowd
    Y. Aksoy, C. Kim, P. Kellnhofer, S. Paris, M. Elgharib, M. Pollefeys, and W. Matusik.
    European Conference on Computer Vision 2018 (ECCV)
    [Manuscript], [Project page]

    Crowd-Guided Ensembles. How Can We Choreograph Crowd Workers for Video Segmentation?
    A. Kasper, G. Patterson, C. Kim, Y. Aksoy, W. Matusik, and M. Elgharib.
    ACM Conference on Human Factors in Computing Systems 2018 (CHI) - Oral
    [Manuscript], [Video teaser], [Project page]

    Video reflection removal through spatio-temporal optimization
    A. Nandoriya*, M. Elgharib*, C. Kim, M. Hefeeda, and W. Matusik. *indicates equal contrbutions.
    International Conference on Computer Vision 2017 (ICCV)
    [Manuscript], [Project page]

    Sports VR Content Generation from Regular Camera Feeds
    K. Calagari, M. Elgharib, S. Shirmohammadi, and M. Hefeeda.
    ACM Multimedia 2017 (ACM MM) - Full Paper
    [Manuscript], [Project page]

    Data Driven 2-D to 3-D Video Conversion for Soccer
    K. Calagari, M. Elgharib, P. Didyk, A. Kaspar, W. Matusik, and M. Hefeeda.
    IEEE Transactions on Multimedia 2017 (TMM)
    [Manuscript], [Project page]

    Painting Style Transfer for Head Portraits using Convolutional Neural Networks
    A. Selim*, M. Elgharib*, and L. Doyle. *joint first author, sorted alphabetically
    ACM Transactions on Graphics (Proceedings of SIGGRAPH 2016)
    Selected as the proceedings back-cover!
    [Manuscript], [Video teaser], [Project page]

    Gradient-based 2-D to 3-D conversion for Soccer Videos
    K. Calagari,M. Elgharib, P. Didyk, A. Kaspar, W. Matusik, and M. Hefeeda.
    ACM Multimedia 2015 (ACM MM) - Oral
    Selected as MIT home-page!
    [Manuscript], [MIT main page], [Project page]

    Video Magnification in Presence of Large Motions
    M. Elgharib, M. Hefeeda, F. Durand and W. T. Freeman.
    Computer Vision and Pattern Recongition 2015 (CVPR)
    [Manuscript], [Project page]

    Retrieval in Long Surveillance Videos using User Described Motion and Object Attributes
    G. Castanon, M. Elgharib, P.M. Jodoin and V. Saligrama.
    IEEE Transactions on Circuits and Systems for Video Technology, 26:12, Dec. 2016 (accepted Aug. 2015) (TCSVT)
    [Manuscript], [Project page]

    Blotch and scratch removal in archived film using a semi-transparent corruption model and a ground-truth generation technique
    M. Elgharib, F. Pitie, and Anil Kokaram.
    EURASIP Journal on Image and Video Processing 2013:33 (EURASIP)
    [Manuscript], [Project page]

    Motion Estimation for Regions of Reflections through Layer Separation
    M. Elgharib, F. Pitie, and Anil Kokaram.
    ACM SIGGRAPH European Conference on Visual Media Production 2011 (CVMP) - Oral
    [Manuscript], [Project page]

    Reflection Detection in Image Sequences
    M. Elgharib, F. Pitie, and Anil Kokaram.
    Computer Vision and Pattern Recognition 2011 (CVPR)
    [Manuscript], [Project page]

    Extraction of Non-binary Blotch Mattes
    M. Elgharib, F. Pitie, and Anil Kokaram.
    International Conference on Image Processing 2009 (ICIP)
    [Manuscript], [Project page]

Technical Reports

  • A. Hassanein, M. Elgharib, A. Selim, S. Bae, M. Hefeeda and W. Matusik, Large-scale, Fast and Accurate Shot Boundary Detection through Spatio-temporal Convolutional Neural Networks, 2017. [Project page] [Dataset] [Code and models]
  • M. Elgharib, Z. Wu, R. Radke and V. Saligarama, Distributed Video Analytics and Anomaly Detection, 2014.
  • Teaching

  • C. Theobalt, M. Elgharib, V. Golyanik, Computer Vision and Machine Learning for Computer Graphics, University of Saarland, 2020.
  • C. Theobalt, M. Elgharib, V. Golyanik, Computer Vision and Machine Learning for Computer Graphics, University of Saarland, 2019.
  • Granted U.S. Patents

  • M. Elgharib, M. Hefeeda, W. Freeman and F. Durand, Method for dynamic video magnification, 2016.
  • M. Hefeeda, K. Calagari, M. Elgharib, W. Matusik, P. Didyk and A. Kaspar, 2D-to-3D video frame conversion, 2016.
  • Previous Positions

  • November 2016 - July 2018: Scientist, Qatar Computing Research Institute (QCRI).
  • October 2013 - November 2016: Post-doc, Qatar Computing Research Institute (QCRI).
  • January 2012 - October 2013: Boston Unversity.
    • Developed reliable and computationally efficient video search techniques for surveillance footage captured in harsh conditions.
  • Education

  • 2011: PhD in Electrical and Computer Engineering, at Trinity College Dublin, Ireland - Full paid non-EU scholarship from IRCSET.
  • 2008: B.A.I in Electrical and Computer Engineering, at Trinity College Dublin, Ireland - Gold Medalist.
  • MISC

  • Reviewer for CVPR, ICCV, ECCV, SIGGAPH, SIGGRAPH-Asia, NeurIPS, PAMI, TIPS, 3DV, ACCV, and others.
  • Links: My Google Scholar, Youtube Channel, Twitter, Linkedin, and Facebook.
  • Hobbies: I enjoy travelling, sightseeing, walking, hiking, natural scenery, watching documentaries and catching up with friends!