Controlling Image Synthesis with Emergent and Designed Priors
Name
chai-lrchai-phd-eecs-2023-thesis.pdf
Description
Thesis PDF
Size
38.02 MB
Format
Adobe PDF
Checksum (MD5)
b448749beb89ce8c126eb0bf93ae6895
Author(s)
Chai, Lucy
Advisor(s)
Isola, Phillip
Date Issued
September 2023
Publisher
Massachusetts Institute of Technology
Abstract
Image synthesis has developed at an unprecedented pace over the past few years, giving us new abilities to create synthetic yet photorealistic content. Typically, unconditional synthesis takes in a tensor of random numbers as input and produces a randomly generated image that mimics real-world content, with little to no way of controlling the result. The work contained in this thesis explores two avenues of obtaining controllable content from image generative models using emergent and designed priors. Emergent priors leverage the capabilities of a pre-trained generator to infer how the world operates, simply by training on large quantities of data. On the other hand, designed priors use built-in constraints to enforce desired properties about the world. Using emergent priors, we can control content by discovering factors of variation and compositional properties in the latent space of synthesis models. We further add coordinate information and camera inputs as designed controls to generate continuous-resolution and 3D-consistent imagery.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
