FaceFolds: Meshed Radiance Manifolds for Efficient Volumetric Rendering of Dynamic Faces
Name
3651304.pdf
Size
12.86 MB
Format
Adobe PDF
Checksum (MD5)
60c23f17078214f08fb331b0ba8e2414
Author(s)
Medin, Safa C.
Li, Gengyan
Du, Ruofei
Garbin, Stephan
Davidson, Philip
Wornell, Gregory W.
Beeler, Thabo
Meka, Abhimitra
Date Issued
May 11, 2024
Journal
Proceedings of the ACM on Computer Graphics and Interactive Techniques
Publisher
Association for Computing Machinery
Citation
Medin, Safa C., Li, Gengyan, Du, Ruofei, Garbin, Stephan, Davidson, Philip et al. 2024. "FaceFolds: Meshed Radiance Manifolds for Efficient Volumetric Rendering of Dynamic Faces." Proceedings of the ACM on Computer Graphics and Interactive Techniques, 7 (1).
Version
Final published version
Abstract
3D rendering of dynamic face captures is a challenging problem, and it demands improvements on several fronts---photorealism, efficiency, compatibility, and configurability. We present a novel representation that enables high-quality volumetric rendering of an actor's dynamic facial performances with minimal compute and memory footprint. It runs natively on commodity graphics soft- and hardware, and allows for a graceful trade-off between quality and efficiency. Our method utilizes recent advances in neural rendering, particularly learning discrete radiance manifolds to sparsely sample the scene to model volumetric effects. We achieve efficient modeling by learning a single set of manifolds for the entire dynamic sequence, while implicitly modeling appearance changes as temporal canonical texture. We export a single layered mesh and view-independent RGBA texture video that is compatible with legacy graphics renderers without additional ML integration. We demonstrate our method by rendering dynamic face captures of real actors in a game engine, at comparable photorealism to state-of-the-art neural rendering techniques at previously unseen frame rates.
MIT Department
Massachusetts Institute of Technology. Signals, Information and Algorithms Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3651304
