MIT Libraries logoDSpace@MIT

MIT
View Item 
  • DSpace@MIT Home
  • MIT Open Access Articles
  • MIT Open Access Articles
  • View Item
  • DSpace@MIT Home
  • MIT Open Access Articles
  • MIT Open Access Articles
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

Non-Linear Monte-Carlo Search in Civilization II

Author(s)
Branavan, Satchuthanan R.; Silver, David; Barzilay, Regina
Thumbnail
DownloadBarzilay-Non-Linear Monte.pdf (294.8Kb)
OPEN_ACCESS_POLICY

Open Access Policy

Creative Commons Attribution-Noncommercial-Share Alike

Terms of use
Creative Commons Attribution-Noncommercial-Share Alike 3.0 http://creativecommons.org/licenses/by-nc-sa/3.0/
Metadata
Show full item record
Abstract
This paper presents a new Monte-Carlo search algorithm for very large sequential decision-making problems. Our approach builds on the recent success of Monte-Carlo tree search algorithms, which estimate the value of states and actions from the mean outcome of random simulations. Instead of using a search tree, we apply non-linear regression, online, to estimate a state-action value function from the outcomes of random simulations. This value function generalizes between related states and actions, and can therefore provide more accurate evaluations after fewer simulations. We apply our Monte-Carlo search algorithm to the game of Civilization II, a challenging multi-agent strategy game with an enormous state space and around $10^{21}$ joint actions. We approximate the value function by a neural network, augmented by linguistic knowledge that is extracted automatically from the official game manual. We show that this non-linear value function is significantly more efficient than a linear value function. Our non-linear Monte-Carlo search wins 80\% of games against the handcrafted, built-in AI for Civilization II.
Date issued
2011-07
URI
http://hdl.handle.net/1721.1/74248
Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Journal
Proceedings of the Twenty-second International Joint Conference on Artificial Intelligence
Publisher
AAAI Press/International Joint Conferences on Artificial Intelligence
Citation
Branavan, S. R. K. David Silver, and Regina Barzilay. "Non-Linear Monte-Carlo Search in Civilization II." in Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, Barcelona, Catalonia, Spain, 16–22 July 2011. p.2404.
Version: Author's final manuscript
ISBN
978-1-57735-512-0
978-1-57735-516-8

Collections
  • MIT Open Access Articles

Browse

All of DSpaceCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

My Account

Login

Statistics

OA StatisticsStatistics by CountryStatistics by Department
MIT Libraries
PrivacyPermissionsAccessibilityContact us
MIT
Content created by the MIT Libraries, CC BY-NC unless otherwise noted. Notify us about copyright concerns.