Convergence Rate of O(1/k) for Optimistic Gradient and Extragradient Methods in Smooth Convex-Concave Saddle Point Problems
Name
19m127375x.pdf
Description
Published version
Size
505.84 KB
Format
Adobe PDF
Checksum (MD5)
9a09060ee20b40fc66843d23bb984c23
Author(s)
Mokhtari, Aryan
Ozdaglar, Asuman E
Pattathil, Sarath
Date Issued
2020
Journal
SIAM Journal on Optimization
Publisher
Society for Industrial & Applied Mathematics (SIAM)
Citation
Mokhtari, Aryan, Ozdaglar, Asuman E and Pattathil, Sarath. 2020. "Convergence Rate of O(1/k) for Optimistic Gradient and Extragradient Methods in Smooth Convex-Concave Saddle Point Problems." SIAM Journal on Optimization, 30 (4).
Version
Final published version
Abstract
© 2020 Society for Industrial and Applied Mathematics We study the iteration complexity of the optimistic gradient descent-ascent (OGDA) method and the extragradient (EG) method for finding a saddle point of a convex-concave unconstrained min-max problem. To do so, we first show that both OGDA and EG can be interpreted as approximate variants of the proximal point method. This is similar to the approach taken in (A. Nemirovski (2004), SIAM J. Optim., 15, pp. 229-251) which analyzes EG as an approximation of the ``conceptual mirror prox."" In this paper, we highlight how gradients used in OGDA and EG try to approximate the gradient of the proximal point method. We then exploit this interpretation to show that both algorithms produce iterates that remain within a bounded set. We further show that the primal-dual gap of the averaged iterates generated by both of these algorithms converge with a rate of O (1/k). Our theoretical analysis is of interest as it provides the first convergence rate estimate for OGDA in the general convex-concave setting. Moreover, it provides a simple convergence analysis for the EG algorithm in terms of function value without using a compactness assumption.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1137/19M127375X
