Greedy Algorithm Almost Dominates in Smoothed Contextual Bandits
Name
19m1247115 (1).pdf
Description
Published version
Size
549.47 KB
Format
Adobe PDF
Checksum (MD5)
4c662d9296abacbbf33674c0a369adb5
Author(s)
Raghavan, Manish
Slivkins, Aleksandrs
Vaughan, Jennifer Wortman
Wu, Zhiwei Steven
Date Issued
April 30, 2023
Journal
SIAM Journal on Computing
Publisher
Society for Industrial & Applied Mathematics (SIAM)
Citation
Raghavan, Manish, Slivkins, Aleksandrs, Vaughan, Jennifer Wortman and Wu, Zhiwei Steven. 2023. "Greedy Algorithm Almost Dominates in Smoothed Contextual Bandits." SIAM Journal on Computing, 52 (2).
Version
Final published version
Abstract
Online learning algorithms, widely used to power search and content optimization onthe web, must balance exploration and exploitation, potentially sacrificing the experience of currentusers in order to gain information that will lead to better decisions in the future. While necessary inthe worst case, explicit exploration has a number of disadvantages compared to the greedy algorithmthat always ``exploits"" by choosing an action that currently looks optimal. We determine under whatconditions inherent diversity in the data makes explicit exploration unnecessary. We build on a recentline of work on the smoothed analysis of the greedy algorithm in the linear contextual bandits model.We improve on prior results to show that the greedy algorithm almost matches the best possibleBayesian regret rate of any other algorithm on the same problem instance whenever the diversityconditions hold. The key technical finding is that data collected by the greedy algorithm sufficesto simulate a run of any other algorithm. Further, we prove that under a particular smoothnessassumption, the Bayesian regret of the greedy algorithm is at most \~O(T 1/3) in the worst case, whereT is the time horizon.
MIT Department
Sloan School of Management
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1137/19M1247115
