A hypothesis for basal ganglia-dependent reinforcement learning in the songbird
Author(s)Fee, Michale S.; Goldberg, Jesse H.
MetadataShow full item record
Most of our motor skills are not innately programmed, but are learned by a combination of motor exploration and performance evaluation, suggesting that they proceed through a reinforcement learning (RL) mechanism. Songbirds have emerged as a model system to study how a complex behavioral sequence can be learned through an RL-like strategy. Interestingly, like motor sequence learning in mammals, song learning in birds requires a basal ganglia (BG)-thalamocortical loop, suggesting common neural mechanisms. Here, we outline a specific working hypothesis for how BG-forebrain circuits could utilize an internally computed reinforcement signal to direct song learning. Our model includes a number of general concepts borrowed from the mammalian BG literature, including a dopaminergic reward prediction error and dopamine-mediated plasticity at corticostriatal synapses. We also invoke a number of conceptual advances arising from recent observations in the songbird. Specifically, there is evidence for a specialized cortical circuit that adds trial-to-trial variability to stereotyped cortical motor programs, and a role for the BG in “biasing” this variability to improve behavioral performance. This BG-dependent “premotor bias” may in turn guide plasticity in downstream cortical synapses to consolidate recently learned song changes. Given the similarity between mammalian and songbird BG-thalamocortical circuits, our model for the role of the BG in this process may have broader relevance to mammalian BG function.
DepartmentMcGovern Institute for Brain Research at MIT
Fee, Michale S. and Jesse H. Goldberg. "A hypothesis for basal ganglia-dependent reinforcement learning in the songbird." Neuroscience 198 (December 2011): 152-170. © 2011 IBRO
Author's final manuscript