No-Regret Learning in General Games
Name
Fishelson-maxfish-SM-EECS-2023-thesis.pdf
Description
Thesis PDF
Size
868.12 KB
Format
Adobe PDF
Checksum (MD5)
841357a3aedfe2ddea95176f2aeb4d4f
Author(s)
Fishelson, Maxwell K.
Advisor(s)
Daskalakis, Constantinos
Date Issued
February 2023
Publisher
Massachusetts Institute of Technology
Abstract
This thesis investigates the regret performance of no-regret learning algorithms in the competitive, though not fully-adversarial, environment of games. We establish exponential improvements on previously best-known external and internal regret bounds for these settings.
We show that Optimistic Hedge – a common variant of multiplicative-weights-updates with recency bias – attains poly(log T) regret in multi-player general-sum games. In particular, when every player of the game uses Optimistic Hedge to iteratively update her strategy in response to the history of play so far, then after T rounds of interaction, each player experiences total regret that is poly(log T). Our bound improves, exponentially, the O(T¹ᐟ²) regret attainable by standard no-regret learners in games, the O(T¹ᐟ⁴) regret attainable by no-regret learners with recency bias [Syr+15], and the O(T¹ᐟ⁶) bound that was recently shown for Optimistic Hedge in the special case of two-player games [CP20]. A corollary of our bound is that Optimistic Hedge converges to coarse correlated equilibrium in general games at a rate of [formula].
We then extend this result from external regret to internal and swap regret, thereby establishing uncoupled learning dynamics that converge to an approximate correlated equilibrium at the rate of [formula]. This substantially improves over the prior best rate of convergence for correlated equilibria of O(T⁻³ᐟ⁴) due to Chen and Peng (NeurIPS ‘20), and it is optimal up to polylogarithmic factors in T.
The results presented here originate from my works [DFG21] and [Ana+22].
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright MIT
Persistent DSpace Link
