On Benefits of Adaptive Variance Inflation in Thompson Sampling
Name
zhu-fengzhu-sm-idss-26-thesis.pdf
Size
3.33 MB
Format
Adobe PDF
Checksum (MD5)
bb504148b82930fbc9d80de1e47aec45
Author(s)
Zhu, Feng
Advisor(s)
Simchi-Levi, David
Date Issued
February 2026
Publisher
Massachusetts Institute of Technology
Abstract
Thompson Sampling (TS) has emerged as a powerful algorithm for sequential decisionmaking, combining strong empirical performance with appealing theoretical guarantees. However, recent work has highlighted that its behavior can deteriorate under stringent safety or robustness requirements — such as controlling the distribution of cumulative regret or maintaining performance under model mis-specification. In this thesis, we address these limitations through the lens of adaptive variance inflation for Thompson Sampling. Our approach introduces a simple, one-line modification: a time- and arm-dependent inflation factor applied to the sampling variance. Despite its simplicity, this adjustment yields several notable benefits. We show that the resulting policy achieves worst-case optimal expected regret together with worst-case optimal, rapidly decaying regret-tail bounds, even under heavy-tailed (sub-exponential) noise or mis-specified reward models. The method is further robust to unknown or mis-specified noise variances, preserving performance without requiring accurate prior knowledge. Beyond cumulative regret, we demonstrate that the policy provides strong post-experiment reliability guarantees: both simple regret and per-arm estimation error satisfy fast-decaying tail bounds, enabling more dependable inference and downstream decision-making. Finally, we extend the approach to settings with unknown, arm-specific noise variances and empirically validate its consistent performance across a wide range of environments. Together, these results show that adaptive variance inflation offers a principled and lightweight way to enhance the safety, robustness, and general applicability of Thompson Sampling.
MIT Department
Massachusetts Institute of Technology. Institute for Data, Systems, and Society
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
