Evidence that recurrent circuits are critical to the ventral stream’s execution of core object recognition behavior
Author(s)Kar, Kohitij; Kubilius, Jonas; Schmidt, Kailyn Marie; Issa, Elias; DiCarlo, James
MetadataShow full item record
Non-recurrent deep convolutional neural networks (CNNs) are currently the best at modeling core object recognition, a behavior that is supported by the densely recurrent primate ventral stream, culminating in the inferior temporal (IT) cortex. If recurrence is critical to this behavior, then primates should outperform feedforward-only deep CNNs for images that require additional recurrent processing beyond the feedforward IT response. Here we first used behavioral methods to discover hundreds of these ‘challenge’ images. Second, using large-scale electrophysiology, we observed that behaviorally sufficient object identity solutions emerged ~30 ms later in the IT cortex for challenge images compared with primate performance-matched ‘control’ images. Third, these behaviorally critical late-phase IT response patterns were poorly predicted by feedforward deep CNN activations. Notably, very-deep CNNs and shallower recurrent CNNs better predicted these late IT responses, suggesting that there is a functional equivalence between additional nonlinear transformations and recurrence. Beyond arguing that recurrent circuits are critical for rapid object identification, our results provide strong constraints for future recurrent model development.
DepartmentMcGovern Institute for Brain Research at MIT; Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences; Center for Brains, Minds, and Machines
Springer Science and Business Media LLC
Kar, Kohitij et al. “Evidence that recurrent circuits are critical to the ventral stream’s execution of core object recognition behavior.” Nature neuroscience, 22, (June 2019): 974–983 © 2019 The Author(s)