Nonlinear conjugate gradient methods: worst-case convergence rates via computer-assisted analyses
Name
10107_2024_2127_ReferencePDF.pdf
Size
1.01 MB
Format
Adobe PDF
Checksum (MD5)
32daf5d4a15dd20eeedafcd81868ff8e
Author(s)
Das Gupta, Shuvomoy
Freund, Robert M.
Sun, Xu A.
Taylor, Adrien
Date Issued
August 22, 2024
Journal
Mathematical Programming
Publisher
Springer Berlin Heidelberg
Citation
Das Gupta, S., Freund, R.M., Sun, X.A. et al. Nonlinear conjugate gradient methods: worst-case convergence rates via computer-assisted analyses. Math. Program. 213, 1–49 (2025).
Version
Author's final manuscript
Abstract
We propose a computer-assisted approach to the analysis of the worst-case convergence of nonlinear conjugate gradient methods (NCGMs). Those methods are known for their generally good empirical performances for large-scale optimization, while having relatively incomplete analyses. Using our computer-assisted approach, we establish novel complexity bounds for the Polak-Ribière-Polyak (PRP) and the Fletcher-Reeves (FR) NCGMs for smooth strongly convex minimization. In particular, we construct mathematical proofs that establish the first non-asymptotic convergence bound for FR (which is historically the first developed NCGM), and a much improved non-asymptotic convergence bound for PRP. Additionally, we provide simple adversarial examples on which these methods do not perform better than gradient descent with exact line search, leaving very little room for improvements on the same class of problems.
MIT Department
Massachusetts Institute of Technology. Operations Research Center
Sloan School of Management
Terms of Use
Creative Commons Attribution-Noncommercial-ShareAlike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1007/s10107-024-02127-7
