Neural representation of action sequences: how far can a simple snippet-matching model take us?
Author(s)
Tan, Cheston; Singer, Jedediah M.; Serre, Thomas; Sheinberg, David; Poggio, Tomaso A.
DownloadNIPS2013.pdf (629.4Kb)
OPEN_ACCESS_POLICY
Open Access Policy
Creative Commons Attribution-Noncommercial-Share Alike
Terms of use
Metadata
Show full item recordAbstract
The macaque Superior Temporal Sulcus (STS) is a brain area that receives and integrates inputs from both the ventral and dorsal visual processing streams (thought to specialize in form and motion processing respectively). For the processing of articulated actions, prior work has shown that even a small population of STS neurons contains sufficient information for the decoding of actor invariant to action, action invariant to actor, as well as the specific conjunction of actor and action. This paper addresses two questions. First, what are the invariance properties of individual neural representations (rather than the population representation) in STS? Second, what are the neural encoding mechanisms that can produce such individual neural representations from streams of pixel images? We find that a baseline model, one that simply computes a linear weighted sum of ventral and dorsal responses to short action “snippets”, produces surprisingly good fits to the neural data. Interestingly, even using inputs from a single stream, both actor-invariance and action-invariance can be produced simply by having different linear weights.
Date issued
2013-12Department
Massachusetts Institute of Technology. Department of Brain and Cognitive SciencesJournal
Advances in Neural Information Processing Systems 26 (NIPS 2013)
Publisher
Neural Information Processing Systems Foundation, Inc.
Citation
Tan, Cheston, Jedediah M. Singer, Thomas Serre, David Sheinberg and Tomaso Poggio. "Neural representation of action sequences: how far can a simple snippet-matching model take us?" Advances in neural information processing systems 26 (NIPS 2013), Lake Tahoe, Nevada, United States, December 5-10, 2013.
Version: Author's final manuscript
ISSN
1049-5258