Sensory processing and categorization in cortical and deep neural networks
Name
nihms-1538926.pdf
Description
Accepted version
Size
1.44 MB
Format
Adobe PDF
Checksum (MD5)
09ee98be3452e04a9a3d4ec14480fc52
Author(s)
Pinotsis, Dimitrios
Miller, Earl K
Date Issued
November 2019
Journal
NeuroImage
Publisher
Elsevier BV
Citation
Pinotsis, Dimitris A. et al. “Sensory processing and categorization in cortical and deep neural networks.” NeuroImage, 202 (November 2019): 116118 © 2019 The Author(s)
Version
Author's final manuscript
Abstract
Many recent advances in artificial intelligence (AI) are rooted in visual neuroscience. However, ideas from more complicated paradigms like decision-making are less used. Although automated decision-making systems are ubiquitous (driverless cars, pilot support systems, medical diagnosis algorithms etc.), achieving human-level performance in decision making tasks is still a challenge. At the same time, these tasks that are hard for AI are easy for humans. Thus, understanding human brain dynamics during these decision-making tasks and modeling them using deep neural networks could improve AI performance. Here we modelled some of the complex neural interactions during a sensorimotor decision making task. We investigated how brain dynamics flexibly represented and distinguished between sensory processing and categorization in two sensory domains: motion direction and color. We used two different approaches for understanding neural representations. We compared brain responses to 1) the geometry of a sensory or category domain (domain selectivity) and 2) predictions from deep neural networks (computation selectivity). Both approaches gave us similar results. This confirmed the validity of our analyses. Using the first approach, we found that neural representations changed depending on context. We then trained deep recurrent neural networks to perform the same tasks as the animals. Using the second approach, we found that computations in different brain areas also changed flexibly depending on context. Color computations appeared to rely more on sensory processing, while motion computations more on abstract categories. Overall, our results shed light to the biological basis of categorization and differences in selectivity and computations in different brain areas. They also suggest a way for studying sensory and categorical representations in the brain: compare brain responses to both a behavioral model and a deep neural network and test if they give similar results.
MIT Department
Picower Institute for Learning and Memory
Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences
Terms of Use
Creative Commons Attribution-NonCommercial-NoDerivs License
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1016/J.NEUROIMAGE.2019.116118
