Bootstrap Thompson Sampling and Sequential Decision Problems in the Behavioral Sciences
Name
2158244019851675.pdf
Description
Published version
Size
693.87 KB
Format
Adobe PDF
Checksum (MD5)
662210c20c924396987a6f20c188dc70
Author(s)
Eckles, Dean
Kaptein, Maurits
Date Issued
2019
Journal
SAGE Open
Publisher
SAGE Publications
Version
Final published version
Abstract
© The Author(s) 2019. Behavioral scientists are increasingly able to conduct randomized experiments in settings that enable rapidly updating probabilities of assignment to treatments (i.e., arms). Thus, many behavioral science experiments can be usefully formulated as sequential decision problems. This article reviews versions of the multiarmed bandit problem with an emphasis on behavioral science applications. One popular method for such problems is Thompson sampling, which is appealing for randomizing assignment and being asymptoticly consistent in selecting the best arm. Here, we show the utility of bootstrap Thompson sampling (BTS), which replaces the posterior distribution with the bootstrap distribution. This often has computational and practical advantages. We illustrate its robustness to model misspecification, which is a common concern in behavioral science applications. We show how BTS can be readily adapted to be robust to dependent data, such as repeated observations of the same units, which is common in behavioral science applications. We use simulations to illustrate parametric Thompson sampling and BTS for Bernoulli bandits, factorial Gaussian bandits, and bandits with repeated observations of the same units.
MIT Department
Sloan School of Management
Terms of Use
Creative Commons Attribution 4.0 International license
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1177/2158244019851675
