| dc.description.abstract | Norepinephrine (NE) is one of the four main neuromodulators in the brain, exerting widespread influence over almost all cortical and subcortical brain regions, with the exception of the striatum. The locus coeruleus (LC) is a small brainstem nucleus, and the primary source of NE in the brain. LC-NE neurons release NE to regulate baseline arousal and to facilitate a variety of sensory-motor and behavioral functions. Dysfunction in the LC-NE system has been implicated in the etiology of ADHD, schizophrenia, anxiety or stress, as well as in the cognitive decline observed in aging and Alzheimer’s disease. Despite its brain-wide effects and established involvement in CNS disorders, much about even the normal function of the LC-NE system in the brain remains unknown. Here, we explore the role of LC-NE in reinforcement learning by studying the spatiotemporal dynamics of transient LC-NE release during learned behaviors, the impact of this release on behavior, and the effects of LC-NE on task representation and processing in cortical target regions. Using a go/no-go task where water-restricted mice must push a lever at a go tone to receive a water reward, and refrain from pushing at a no-go tone to avoid an air-puff punishment, we first explored the role of LC-NE in this behavior. Additionally, we used tones of differing intensity to modulate the uncertainty on a trial by trial basis. We found that LC-NE is important for two aspects of reinforcement learning: promoting task execution under uncertain conditions and facilitating improved performance after a surprising outcome. In line with these results, using both opto-tagging and 2-photon imaging we found that LC-NE neurons are active during two task epochs, pre-lever press and post-reinforcement. Pre-press, phasic LC-NE neuronal activity scaled with the degree of certainty of the tone identity, while post-reinforcement LC-NE activity scaled with the degree of surprise. Thus, reward following ‘hits’ after low intensity go tones elicited higher activity than high intensity go tones, and punishment following movement after no-go tones, or false alarms, elicited the highest activity. Our electrophysiology data also indicated some degree of heterogeneity in LC-NE neuronal activity, such that some neurons exhibited pre-press activity, some had post-reward activity, few had both, but all LC-NE neurons responded strongly after a false alarm. To further explore this heterogeneity in task encoding, we performed 2-photon imaging of LC-NE axonal calcium activity during the task in two target regions, the motor and prefrontal cortices. We found that pre-press activity is preferentially sent to the motor cortex to facilitate task execution, while post-punishment activity is projected to both regions equally. To determine how LC-NE affects ongoing processing in target regions, we performed high density single unit recordings in motor and prefrontal cortices while silencing the LC on a subset of trials. We found that LC-NE activity following an air-puff punishment changed population activity in the cortex such that the population representation of the stimulus on the subsequent trial is more discriminable (i.e. elicited larger difference in population trajectories between go and no-go tones). To explore how this false alarm signal might be sustained to have lasting effects on behavior through the next trial (seconds later), we also studied the role of astrocytes in our reinforcement learning task. Astrocytes are the most abundant non-neuronal cell type in the brain and have been suggested to both reflect and modulate neuronal activity. They operate on multiple timescales and have been shown to be responsive to NE in the brain, and thus present a viable means by which LC-NE learning signals can have lasting impacts on population activity and ongoing behavior. Using 2-photon imaging of astrocyte calcium, we found that astrocytes show reliable and long-lasting increases in calcium activity following an air-puff punishment. Manipulating astrocyte calcium dynamics in prefrontal and motor cortex abolished the improvement in behavioral performance that typically follows a false alarm, indicating that astrocyte signaling mediates improvement in performance following a surprising outcome. Finally, to determine whether LC-NE is acting directly on astrocytes to mediate reinforcement learning, we performed 2-photon imaging of astrocyte and neuronal calcium in the cortex while chemogenetically silencing LC neurons. We found that without LC-NE activity, cortical astrocytes no longer exhibit the typical sustained increase in calcium following a false alarm. Taken together, these data indicate that LC-NE neurons signal critical components of reinforcement learning, including reward uncertainty and reward prediction error or surprise, and selectively project these signals throughout the brain to facilitate reinforcement learning by altering astrocyte calcium and neuronal population activity in the cortex. | |
