MOST-GAN: 3D Morphable StyleGAN for Disentangled Face Image Manipulation
Name
2111.01048.pdf
Description
Submitted version
Size
46.19 MB
Format
Adobe PDF
Checksum (MD5)
f325a47dd23617aec4db378f1d5b0f53
Author(s)
Medin, Safa C
Egger, Bernhard
Cherian, Anoop
Wang, Ye
Tenenbaum, Joshua B
Liu, Xiaoming
Marks, Tim K
Date Issued
2022
Journal
Proceedings of the AAAI Conference on Artificial Intelligence
Publisher
Association for the Advancement of Artificial Intelligence (AAAI)
Citation
Medin, Safa C, Egger, Bernhard, Cherian, Anoop, Wang, Ye, Tenenbaum, Joshua B et al. 2022. "MOST-GAN: 3D Morphable StyleGAN for Disentangled Face Image Manipulation." Proceedings of the AAAI Conference on Artificial Intelligence, 36 (2).
Version
Original manuscript
Abstract
Recent advances in generative adversarial networks (GANs) have led to remarkable achievements in face image synthesis. While methods that use style-based GANs can generate strikingly photorealistic face images, it is often difficult to control the characteristics of the generated faces in a meaningful and disentangled way. Prior approaches aim to achieve such semantic control and disentanglement within the latent space of a previously trained GAN. In contrast, we propose a framework that a priori models physical attributes of the face such as 3D shape, albedo, pose, and lighting explicitly, thus providing disentanglement by design. Our method, MOST-GAN, integrates the expressive power and photorealism of style-based GANs with the physical disentanglement and flexibility of nonlinear 3D morphable models, which we couple with a state-of-the-art 2D hair manipulation network. MOST-GAN achieves photorealistic manipulation of portrait images with fully disentangled 3D control over their physical attributes, enabling extreme manipulation of lighting, facial expression, and pose variations up to full profile view.
MIT Department
Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1609/AAAI.V36I2.20091
