Seeing What a GAN Cannot Generate
Name
1910.11626.pdf
Description
Accepted version
Size
5.79 MB
Format
Unknown
Checksum (MD5)
e5d3ae2bd4fc5230c672b2b9e7453b11
Author(s)
Bau, David
Zhu, Jun-Yan
Wulff, Jonas
Peebles, William
Strobelt, Hendrik
Torralba, Antonio
Date Issued
October 2020
Journal
Proceedings of the IEEE International Conference on Computer Vision
Publisher
IEEE
Citation
2020. "Seeing What a GAN Cannot Generate." Proceedings of the IEEE International Conference on Computer Vision, 2019-October.
Version
Author's final manuscript
Abstract
© 2019 IEEE. Despite the success of Generative Adversarial Networks (GANs), mode collapse remains a serious issue during GAN training. To date, little work has focused on understanding and quantifying which modes have been dropped by a model. In this work, we visualize mode collapse at both the distribution level and the instance level. First, we deploy a semantic segmentation network to compare the distribution of segmented objects in the generated images with the target distribution in the training set. Differences in statistics reveal object classes that are omitted by a GAN. Second, given the identified omitted object classes, we visualize the GAN's omissions directly. In particular, we compare specific differences between individual photos and their approximate inversions by a GAN. To this end, we relax the problem of inversion and solve the tractable problem of inverting a GAN layer instead of the entire generator. Finally, we use this framework to analyze several recent GANs trained on multiple datasets and identify their typical failure cases.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/ICCV.2019.00460
