Reinforcement biases subsequent perceptual decisions when confidence is low, a widespread behavioral phenomenon
Name
elife-49834-v3.pdf
Description
Published version
Size
1.11 MB
Format
Adobe PDF
Checksum (MD5)
bbbe01892350b861c4db90edc35ca860
Author(s)
Lak, Armin
Hueske, Emily
Hirokawa, Junya
Masset, Paul
Ott, Torben
Urai, Anne E
Donner, Tobias H
Carandini, Matteo
Tonegawa, Susumu
Uchida, Naoshige
Date Issued
2020
Journal
eLife
Publisher
eLife Sciences Publications, Ltd
Version
Final published version
Abstract
© 2020, eLife Sciences Publications Ltd. All rights reserved. Learning from successes and failures often improves the quality of subsequent decisions. Past outcomes, however, should not influence purely perceptual decisions after task acquisition is complete since these are designed so that only sensory evidence determines the correct choice. Yet, numerous studies report that outcomes can bias perceptual decisions, causing spurious changes in choice behavior without improving accuracy. Here we show that the effects of reward on perceptual decisions are principled: past rewards bias future choices specifically when previous choice was difficult and hence decision confidence was low. We identified this phenomenon in six datasets from four laboratories, across mice, rats, and humans, and sensory modalities from olfaction and audition to vision. We show that this choice-updating strategy can be explained by reinforcement learning models incorporating statistical decision confidence into their teaching signals. Thus, despite being suboptimal from the experimenter’s perspective, confidence-guided reinforcement learning optimizes behavior in uncertain, real-world situations.
MIT Department
Picower Institute for Learning and Memory
Massachusetts Institute of Technology. Department of Biology
Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences
McGovern Institute for Brain Research at MIT
Howard Hughes Medical Institute
Terms of Use
Creative Commons Attribution 4.0 International license
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.7554/ELIFE.49834
