Factor-√2 Acceleration of Accelerated Gradient Methods
Name
245_2023_10047_ReferencePDF.pdf
Size
814.55 KB
Format
Adobe PDF
Checksum (MD5)
0a4c273b93ba3ec2fc409458057bd284
Author(s)
Park, Chanwoo
Park, Jisun
Ryu, Ernest K.
Date Issued
August 23, 2023
Publisher
Springer US
Citation
Applied Mathematics & Optimization. 2023 Aug 23;88(3):77
Version
Author's final manuscript
Abstract
Abstract
The optimized gradient method (OGM) provides a factor-
$$\sqrt{2}$$
2
speedup upon Nesterov’s celebrated accelerated gradient method in the convex (but non-strongly convex) setup. However, this improved acceleration mechanism has not been well understood; prior analyses of OGM relied on a computer-assisted proof methodology, so the proofs were opaque for humans despite being verifiable and correct. In this work, we present a new analysis of OGM based on a Lyapunov function and linear coupling. These analyses are developed and presented without the assistance of computers and are understandable by humans. Furthermore, we generalize OGM’s acceleration mechanism and obtain a factor-
$$\sqrt{2}$$
2
speedup in other setups: acceleration with a simpler rational stepsize, the strongly convex setup, and the mirror descent setup.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1007/s00245-023-10047-9
