Last-iterate convergence: Zero-sum games and constrained min-max optimization
Name
LIPIcs-ITCS-2019-27.pdf
Description
Published version
Size
631.48 KB
Format
Adobe PDF
Checksum (MD5)
1ab9522e2a5f2bacf06bd9c7391bc1fe
Author(s)
Daskalakis, C
Panageas, I
Date Issued
January 1, 2019
Journal
Leibniz International Proceedings in Informatics, LIPIcs
Citation
Daskalakis, C and Panageas, I. 2019. "Last-iterate convergence: Zero-sum games and constrained min-max optimization." Leibniz International Proceedings in Informatics, LIPIcs, 124.
Version
Final published version
Abstract
© Constantinos Daskalakis and Ioannis Panageas. Motivated by applications in Game Theory, Optimization, and Generative Adversarial Networks, recent work of Daskalakis et al [Daskalakis et al., ICLR, 2018] and follow-up work of Liang and Stokes [Liang and Stokes, 2018] have established that a variant of the widely used Gradient Descent/Ascent procedure, called “Optimistic Gradient Descent/Ascent (OGDA)”, exhibits last-iterate convergence to saddle points in unconstrained convex-concave min-max optimization problems. We show that the same holds true in the more general problem of constrained min-max optimization under a variant of the no-regret Multiplicative-Weights-Update method called “Optimistic Multiplicative-Weights Update (OMWU)”. This answers an open question of Syrgkanis et al [Syrgkanis et al., NIPS, 2015]. The proof of our result requires fundamentally different techniques from those that exist in no-regret learning literature and the aforementioned papers. We show that OMWU monotonically improves the Kullback-Leibler divergence of the current iterate to the (appropriately normalized) min-max solution until it enters a neighborhood of the solution. Inside that neighborhood we show that OMWU becomes a contracting map converging to the exact solution. We believe that our techniques will be useful in the analysis of the last iterate of other learning algorithms.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Creative Commons Attribution 3.0 unported license
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.4230/LIPIcs.ITCS.2019.27
