Nonasymptotic Analysis of Monte Carlo Tree Search
Name
1902.05213.pdf
Description
Submitted version
Size
1.32 MB
Format
Adobe PDF
Checksum (MD5)
d9925731692bea04b1620fa66ce285b1
Author(s)
Shah, Devavrat
Xie, Qiaomin
Xu, Zhi
Date Issued
March 1, 2022
Journal
Operations Research
Publisher
Institute for Operations Research and the Management Sciences (INFORMS)
Citation
Shah, Devavrat, Xie, Qiaomin and Xu, Zhi. 2022. "Nonasymptotic Analysis of Monte Carlo Tree Search." Operations Research.
Version
Original manuscript
Abstract
In “Nonasymptotic Analysis of Monte Carlo Tree Search,” D. Shah, Q. Xie, and Z. Xu consider the popular tree-based search strategy, the Monte Carlo Tree Search (MCTS), in the context of the infinite-horizon discounted Markov decision process. They show that MCTS with an appropriate polynomial rather than logarithmic bonus term indeed leads to the desired convergence property. The authors derive the results by establishing a polynomial concentration property of regret for a class of nonstationary multiarm bandits. Furthermore, using this as a building block, they demonstrate that MCTS, combined with nearest neighbor supervised learning, acts as a “policy improvement” operator that can iteratively improve value function approximation.
MIT Department
Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1287/opre.2021.2239
