2026-09-19T17:17:14Zhttps://dspace.mit.edu/server/oai/requestoai:dspace.mit.edu:1721.1/1589222025-04-07T09:27:19Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023
Ozdaglar, Asuman
Farina, Gabriele
Liu, Mingyang
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
2025-03-27T16:58:22Z
2025-03-27T16:58:22Z
2025-02
2025-03-04T17:28:56.371Z
https://hdl.handle.net/1721.1/158922
In this thesis, we explore the design of algorithms capable of handling large games where the state space is too large to store strategies in a tabular format from a theoretical perspective. Specifically, we focus on developing algorithms suitable for deep reinforcement learning in two-player zero-sum extensive-form games. There are three critical properties for effective deep multi-agent reinforcement learning: (last/best) iterate convergence, efficient utilization of stochastic trajectory feedback, and theoretically sound avoidance of importance sampling corrections. Chapter 3 introduces Regularized Optimistic Mirror Descent (Reg-OMD), which provably converges to the Nash equilibrium (NE) linearly in last-iterate. Chapter 4 shows that algorithms based on regret decomposition enjoy best-iterate convergence to the NE. Chapter 5 proposes Q-value based Regret Minimization (QFR), which achieves all three properties simultaneously.
S.M.
Massachusetts Institute of Technology
Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)
Copyright retained by author(s)
https://creativecommons.org/licenses/by-nc-nd/4.0/
On Solving Larger Games: Designing New Algorithms Adaptable to Deep Reinforcement Learning
Thesis
application/pdf
Master
Master of Science in Electrical Engineering and Computer Science
Publication
unknown
unknown
<Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="5cde68f3-e0ed-40ef-bb86-cfdbced4ab98">
<Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type>
<Title>On Solving Larger Games: Designing New Algorithms Adaptable to Deep Reinforcement Learning</Title>
<PublishedIn>
<Publication>
</Publication>
</PublishedIn>
<PublicationDate>2025-02</PublicationDate>
<Authors>
<Author>
<DisplayName>Liu, Mingyang</DisplayName>
<Affiliation>
<OrgUnit>
</OrgUnit>
</Affiliation>
</Author>
</Authors>
<Editors>
</Editors>
<Publishers>
<Publisher>
<DisplayName>Massachusetts Institute of Technology</DisplayName>
<OrgUnit />
</Publisher>
</Publishers>
<License>https://creativecommons.org/licenses/by-nc-nd/4.0/</License>
<Abstract>In this thesis, we explore the design of algorithms capable of handling large games where the state space is too large to store strategies in a tabular format from a theoretical perspective. Specifically, we focus on developing algorithms suitable for deep reinforcement learning in two-player zero-sum extensive-form games. There are three critical properties for effective deep multi-agent reinforcement learning: (last/best) iterate convergence, efficient utilization of stochastic trajectory feedback, and theoretically sound avoidance of importance sampling corrections. Chapter 3 introduces Regularized Optimistic Mirror Descent (Reg-OMD), which provably converges to the Nash equilibrium (NE) linearly in last-iterate. Chapter 4 shows that algorithms based on regret decomposition enjoy best-iterate convergence to the NE. Chapter 5 proposes Q-value based Regret Minimization (QFR), which achieves all three properties simultaneously.</Abstract>
<Access xmlns="http://purl.org/coar/access_right"
>
</Access>
</Publication>