Challenging the Classical View: Recognition of Identity and Expression as Integrated Processes
Name
brainsci-13-00296.pdf
Size
3.5 MB
Format
Adobe PDF
Checksum (MD5)
d742913d9b2248722b820c2c5addbd22
Author(s)
Schwartz, Emily
O’Nell, Kathryn
Saxe, Rebecca
Anzellotti, Stefano
Date Issued
February 10, 2023
Publisher
Multidisciplinary Digital Publishing Institute
Citation
Brain Sciences 13 (2): 296 (2023)
Version
Final published version
Abstract
Recent neuroimaging evidence challenges the classical view that face identity and facial expression are processed by segregated neural pathways, showing that information about identity and expression are encoded within common brain regions. This article tests the hypothesis that integrated representations of identity and expression arise spontaneously within deep neural networks. A subset of the CelebA dataset is used to train a deep convolutional neural network (DCNN) to label face identity (chance = 0.06%, accuracy = 26.5%), and the FER2013 dataset is used to train a DCNN to label facial expression (chance = 14.2%, accuracy = 63.5%). The identity-trained and expression-trained networks each successfully transfer to labeling both face identity and facial expression on the Karolinska Directed Emotional Faces dataset. This study demonstrates that DCNNs trained to recognize face identity and DCNNs trained to recognize facial expression spontaneously develop representations of facial expression and face identity, respectively. Furthermore, a congruence coefficient analysis reveals that features distinguishing between identities and features distinguishing between expressions become increasingly orthogonal from layer to layer, suggesting that deep neural networks disentangle representational subspaces corresponding to different sources.
MIT Department
Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.3390/brainsci13020296
