Repository logo
Log in(current)
Repository logoMIT Open ScholarshipDSpace@MIT
  1. Home
  2. MIT Open Access Articles
  3. MIT Open Access Articles
  4. Fictitious Play in Zero-Sum Stochastic Games

Fictitious Play in Zero-Sum Stochastic Games

Thumbnail Image
Download
Name

21m1426675.pdf

Description
Published version
Size

472.63 KB

Format

Adobe PDF

Checksum (MD5)

e1ae226c494e3ef19ec1c87299067aa3

Author(s)
Sayin, Muhammed O
•
Parise, Francesca
•
Ozdaglar, Asuman
Date Issued
August 2022
Journal
SIAM Journal on Control and Optimization
Publisher
Society for Industrial & Applied Mathematics (SIAM)
Version
Final published version
Abstract
We present a novel variant of fictitious play dynamics combining classical fictitiousplay with Q-learning for stochastic games and analyze its convergence properties in two-player zero-sum stochastic games. Our dynamics involves players forming beliefs on the opponent strategyand their own continuation payoff (Q-function), and playing a greedy best response by using theestimated continuation payoffs. Players update their beliefs from observations of opponent actions.A key property of the learning dynamics is that update of the beliefs on Q-functions occurs at aslower timescale than update of the beliefs on strategies. We show that in both the model-based andmodel-free cases (without knowledge of player payoff functions and state transition probabilities),the beliefs on strategies converge to a stationary mixed Nash equilibrium of the zero-sum stochasticgame.
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
https://dspace.mit.edu/handle/1721.1/167980
DOI of Published Version
https://doi.org/10.1137/21M1426675
Repository logo
PrivacyPermissionsAccessibilityContact us
Repository logo
Notify us about copyright concerns.