MIT Libraries logoDSpace@MIT

MIT
View Item 
  • DSpace@MIT Home
  • MIT Libraries
  • MIT Theses
  • Graduate Theses
  • View Item
  • DSpace@MIT Home
  • MIT Libraries
  • MIT Theses
  • Graduate Theses
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

High resolution neural frontal face synthesis from face encodings using adversarial loss

Author(s)
Wang, Andy(Andy L.),M. Eng.Massachusetts Institute of Technology.
Thumbnail
Download1128187166-MIT.pdf (14.91Mb)
Other Contributors
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.
Advisor
Wojciech Matusik.
Terms of use
MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission. http://dspace.mit.edu/handle/1721.1/7582
Metadata
Show full item record
Abstract
In this thesis, we present a novel neural network method to synthesize a person's face imagery with frontal face and neutral expression, given a single unconstrained face photograph. We achieve this by a data-driven approach to train neural networks with a large-scale in-the-wild dataset of face images. The most common way to tackle this is supervised learning, which requires many ground-truth input-output pairs. Moreover, in our problem context, finding clean frontal and neutral expression faces without occlusions leads to other challenging problems. To avoid this, we take a neural knowledge transfer approach, where we first train modular networks for each well-defined sub-task and exploit them to instill semantic senses to train the face decoder, i.e., neutral face synthesizer. For sub-tasks, we utilize face landmark detection and recognition modules, where curated datasets exist. In particular, the face recognition sub-task learns features strongly invariant to lighting, pose, and facial expression variations. Given the recognition feature, we leverage this invariance to train our face decoder to produce consistent frontal and neutral expression faces, while constraining each generated face: 1) to be a forward facing pose using the network trained for the landmark detection, and 2) to preserve the same identity as the input face using the network trained for face recognition. Furthermore, we attempt to boost the realism of the output faces using adversarial loss, in which a discriminator competes with the generator network and guides the generation of higher quality faces. In test time, only the face recognition network and face decoder are used to synthesize neutral faces. Our approach does not require supervised data and further minimizes sensitive data pre-processing pipelines. Compared to competing fully-supervised methods, our method produces comparable or often even favorable face appearances.
Description
This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.
 
Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2019
 
Cataloged from student-submitted PDF version of thesis.
 
Includes bibliographical references (pages 49-51).
 
Date issued
2019
URI
https://hdl.handle.net/1721.1/123120
Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Publisher
Massachusetts Institute of Technology
Keywords
Electrical Engineering and Computer Science.

Collections
  • Graduate Theses

Browse

All of DSpaceCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

My Account

Login

Statistics

OA StatisticsStatistics by CountryStatistics by Department
MIT Libraries
PrivacyPermissionsAccessibilityContact us
MIT
Content created by the MIT Libraries, CC BY-NC unless otherwise noted. Notify us about copyright concerns.