Metamers of neural networks reveal divergence from human perceptual systems
Name
NeurIPS-2019-metamers-of-neural-networks-reveal-divergence-from-human-perceptual-systems-Paper.pdf
Description
Published version
Size
2.07 MB
Format
Unknown
Checksum (MD5)
39d1cbe335622dc323cd88a517ec8399
Author(s)
Feather, Jenelle
Durango, Alex
Gonzalez, Ray
McDermott, Joshua Hartman
Date Issued
December 2019
Journal
Advances in Neural Information Processing Systems
Citation
2019. "Metamers of neural networks reveal divergence from human perceptual systems." Advances in Neural Information Processing Systems, 32.
Version
Final published version
Abstract
© 2019 Neural information processing systems foundation. All rights reserved. Deep neural networks have been embraced as models of sensory systems, instantiating representational transformations that appear to resemble those in the visual and auditory systems. To more thoroughly investigate their similarity to biological systems, we synthesized model metamers - stimuli that produce the same responses at some stage of a network's representation. We generated model metamers for natural stimuli by performing gradient descent on a noise signal, matching the responses of individual layers of image and audio networks to a natural image or speech signal. The resulting signals reflect the invariances instantiated in the network up to the matched layer. We then measured whether model metamers were recognizable to human observers - a necessary condition for the model representations to replicate those of humans. Although model metamers from early network layers were recognizable to humans, those from deeper layers were not. Auditory model metamers became more human-recognizable with architectural modifications that reduced aliasing from pooling operations, but those from the deepest layers remained unrecognizable. We also used the metamer test to compare model representations. Cross-model metamer recognition dropped off for deeper layers, roughly at the same point that human recognition deteriorated, indicating divergence across model representations. The results reveal discrepancies between model and human representations, but also show how metamers can help guide model refinement and elucidate model representations.
MIT Department
Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences
McGovern Institute for Brain Research at MIT
Center for Brains, Minds, and Machines
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://papers.nips.cc/paper/2019/hash/ac27b77292582bc293a51055bfc994ee-Abstract.html
