New computational guarantees for solving convex optimization problems with first order methods, via a function growth condition measure
Name
10107_2017_1164_ReferencePDF.pdf
Size
292.29 KB
Format
Adobe PDF
Checksum (MD5)
43b5ab9221c888c0a34e3bd6bb4f6ece
Author(s)
Freund, Robert Michael
Lu, Haihao
Date Issued
June 2017
Journal
Mathematical Programming
Publisher
Springer Berlin Heidelberg
Citation
Freund, Robert M., and Haihao Lu. “New Computational Guarantees for Solving Convex Optimization Problems with First Order Methods, via a Function Growth Condition Measure.” Mathematical Programming, vol. 170, no. 2, Aug. 2018, pp. 445–77.
Version
Author's final manuscript
Abstract
Motivated by recent work of Renegar, we present new computational methods and associated computational guarantees for solving convex optimization problems using first-order methods. Our problem of interest is the general convex optimization problem f[superscript ∗] = min[subscript x∈Q] f(x),where we presume knowledge of a strict lower bound f[subscript slb] < f[superscript ∗]. [Indeed, f[subscript slb] is naturally known
when optimizing many loss functions in statistics and machine learning (least-squares, logistic loss, exponential loss, total variation loss, etc.) as well as in Renegar’s transformed version of the standard conic optimization problem; in all these cases one has f[subscript slb] = 0 < f[superscript ∗]
.] We introduce a new functional measure called the growth constant G for f(·), that measures how quickly the level sets of f(·) grow relative to the function value, and that plays a fundamental role in the complexity analysis. When f(·) is non-smooth, we present new computational guarantees for the Subgradient Descent Method and for smoothing methods, that can improve existing computational guarantees in several ways, most notably when the initial iterate x[superscript 0] is far from the optimal solution set. When f(·) is smooth, we present a scheme for periodically restarting the Accelerated Gradient Method that can also improve existing computational guarantees when x[superscript 0] is far from the optimal solution set, and in the presence of added structure we present a scheme using parametrically increased smoothing that further improves the associated computational
guarantees
MIT Department
Massachusetts Institute of Technology. Department of Mathematics
Sloan School of Management
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1007/s10107-017-1164-1
