3DPR: Single Image 3D Portrait Relighting with Generative Priors
Name
3757377.3763962.pdf
Size
55.11 MB
Format
Adobe PDF
Checksum (MD5)
a093c810a6adb3d7551bea93587386b2
Author(s)
Rao, Pramod
Meka, Abhimitra
Zhou, Xilong
Fox, Gereon
B R, Mallikarjun
Zhan, Fangneng
Weyrich, Tim
Bickel, Bernd
Pfister, Hanspeter
Matusik, Wojciech
Date Issued
December 14, 2025
Publisher
ACM|SIGGRAPH Asia 2025 Conference Papers
Citation
Pramod Rao, Abhimitra Meka, Xilong Zhou, Gereon Fox, Mallikarjun B R, Fangneng Zhan, Tim Weyrich, Bernd Bickel, Hanspeter Pfister, Wojciech Matusik, Thabo Beeler, Mohamed Elgharib, Marc Habermann, and Christian Theobalt. 2025. 3DPR: Single Image 3D Portrait Relighting with Generative Priors. In Proceedings of the SIGGRAPH Asia 2025 Conference Papers (SA Conference Papers '25). Association for Computing Machinery, New York, NY, USA, Article 108, 1–12.
Version
Final published version
Abstract
Rendering novel, relit views of a human head, given a monocular portrait image as input, is an inherently underconstrained problem. The traditional graphics solution is to explicitly decompose the input image into geometry, material and lighting via differentiable rendering; but this is constrained by the multiple assumptions and approximations of the underlying models and parameterizations of these scene components. We propose 3DPR, an image-based relighting model that leverages generative priors learnt from multi-view One-Light-at-A-Time (OLAT) images captured in a light stage. We introduce a new diverse and large-scale multi-view 4K OLAT dataset of 139 subjects to learn a high-quality prior over the distribution of high-frequency face reflectance. We leverage the latent space of a pre-trained generative head model that provides a rich prior over face geometry learnt from in-the-wild image datasets. The input portrait is first embedded in the latent manifold of such a model through an encoder-based inversion process. Then a novel triplane-based reflectance network trained on our lightstage data is used to synthesize high-fidelity OLAT images to enable image-based relighting. Our reflectance network operates in the latent space of the generative head model, crucially enabling a relatively small number of lightstage images to train the reflectance model. Combining the generated OLATs according to a given HDRI environment maps yields physically accurate environmental relighting results. Through quantitative and qualitative evaluations, we demonstrate that 3DPR outperforms previous methods, particularly in preserving identity and in capturing lighting effects such as specularities, self-shadows, and subsurface scattering.
Description
SA Conference Papers ’25, December 15–18, 2025, Hong Kong, Hong Kong
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Creative Commons Attribution-NonCommercial
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3757377.3763962
