GAN dissection: Visualizing and understanding generative adversarial networks
Name
850f5beb9fbbedefecd6a9a8753e693cc9a0aa37.pdf
Description
Accepted version
Size
7.02 MB
Format
Adobe PDF
Checksum (MD5)
928df50d5c2c07fed139426f7caf0a95
Author(s)
Bau, David
Zhu, Junyan
Tenenbaum, Joshua B
Freeman, William T
Torralba, Antonio
Date Issued
May 2019
Journal
ICLR 2019 International Conference on Learning Representations
Publisher
International Society of the Learning Sciences
Citation
Bau, David et al. “GAN dissection: Visualizing and understanding generative adversarial networks.” Paper presented at the ICLR 2019 International Conference on Learning Representations, New Orleans, Louisiana, May 6-9, 2019, International Society of the Learning Sciences © 2019 The Author(s)
Version
Author's final manuscript
Abstract
All Rights Reserved. Generative Adversarial Networks (GANs) have recently achieved impressive results for many real-world applications, and many GAN variants have emerged with improvements in sample quality and training stability. However, they have not been well visualized or understood. How does a GAN represent our visual world internally? What causes the artifacts in GAN results? How do architectural choices affect GAN learning? Answering such questions could enable us to develop new insights and better models. In this work, we present an analytic framework to visualize and understand GANs at the unit-, object-, and scene-level. We first identify a group of interpretable units that are closely related to object concepts using a segmentation-based network dissection method. Then, we quantify the causal effect of interpretable units by measuring the ability of interventions to control objects in the output. We examine the contextual relationship between these units and their surroundings by inserting the discovered object concepts into new images. We show several practical applications enabled by our framework, from comparing internal representations across different layers, models, and datasets, to improving GANs by locating and removing artifact-causing units, to interactively manipulating objects in a scene. We provide open source interpretation tools to help researchers and practitioners better understand their GAN models.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://openreview.net/forum?id=Hyg_X2C5FX
