On the Complexity of Nonconvex-Strongly-Concave Smooth Minimax Optimization Using First-Order Methods
Name
Li-haochuan-SM-EECS-2021-thesis.pdf
Description
Thesis PDF
Size
613.82 KB
Format
Adobe PDF
Checksum (MD5)
c7c97a964e7cd122acc74b3bfaccc501
Author(s)
Li, Haochuan
Advisor(s)
Jadbabaie, Ali
Rakhlin, Alexander
Date Issued
June 2021
Publisher
Massachusetts Institute of Technology
Abstract
The problem of minimax optimization arises in a wide range of applications. When the objective function is convex-concave, almost the full picture is known. However, the general nonconvex-concave setting is less understood. In this work, we study the complexity of nonconvex-strongly-concave minimax optimization using first-order methods. First, we provide a first-order oracle complexity lower bound for finding stationary points of nonconvex-strongly-concave smooth min-max optimization problems. We establish a lower bound of ฮฉ ( โ ๐
๐โปยฒ) for deterministic oracles, where ๐ defines the level of approximate stationarity and ๐
is the condition number, which matches the existing upper bound achieved in (Lin et al., 2020b) up to logarithmic factors. For stochastic oracles, we provide a lower bound of ฮฉ (๏ธโ ๐
๐โปยฒ + ๐
ยน/ยณ ๐ โปโด)๏ธ . Second, we study the specific first-order algorithm, gradient descent-ascent (GDA). We show that for quadratic or nearly quadratic nonconvex-strongly-concave functions under our assumptions, two-time-scale GDA with appropriate stepsizes achieves a linear convergence rate. Then we also extend our result to stochastic gradient descent-ascent (SGDA).
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright MIT
Persistent DSpace Link
