On Solving Larger Games: Designing New Algorithms Adaptable to Deep Reinforcement Learning
Name
liu-liumy19-sm-eecs-2025-thesis.pdf
Description
Thesis PDF
Size
1.02 MB
Format
Adobe PDF
Checksum (MD5)
f127a3e7e1c5bce078336ecb9de1023d
Author(s)
Liu, Mingyang
Advisor(s)
Ozdaglar, Asuman
Farina, Gabriele
Date Issued
February 2025
Publisher
Massachusetts Institute of Technology
Abstract
In this thesis, we explore the design of algorithms capable of handling large games where the state space is too large to store strategies in a tabular format from a theoretical perspective. Specifically, we focus on developing algorithms suitable for deep reinforcement learning in two-player zero-sum extensive-form games. There are three critical properties for effective deep multi-agent reinforcement learning: (last/best) iterate convergence, efficient utilization of stochastic trajectory feedback, and theoretically sound avoidance of importance sampling corrections. Chapter 3 introduces Regularized Optimistic Mirror Descent (Reg-OMD), which provably converges to the Nash equilibrium (NE) linearly in last-iterate. Chapter 4 shows that algorithms based on regret decomposition enjoy best-iterate convergence to the NE. Chapter 5 proposes Q-value based Regret Minimization (QFR), which achieves all three properties simultaneously.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)
Copyright retained by author(s)
Persistent DSpace Link
