Relatively Smooth Convex Optimization by First-Order Methods, and Applications
Author(s)
Nesterov, Yurii; Lu, Haihao; Freund, Robert Michael
Download16m1099546.pdf (280.3Kb)
PUBLISHER_POLICY
Publisher Policy
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Terms of use
Metadata
Show full item recordAbstract
The usual approach to developing and analyzing first-order methods for smooth convex optimization assumes that the gradient of the objective function is uniformly smooth with some Lipschitz constant L. However, in many settings the differentiable convex function f(?) is not uniformly smooth-for example, in D-optimal design where f(x) := -ln det(HXHT) and X := Diag(x), or even the univariate setting with f(x) := -ln(x)+x2. In this paper we develop a notion of "relative smoothness" and relative strong convexity that is determined relative to a user-specified "reference function" h(?) (that should be computationally tractable for algorithms), and we show that many differentiable convex functions are relatively smooth with respect to a correspondingly fairly simple reference function h(?). We extend two standard algorithms-the primal gradient scheme and the dual averaging scheme-to our new setting, with associated computational guarantees. We apply our new approach to develop a new first-order method for the D-optimal design problem, with associated computational complexity analysis. Some of our results have a certain overlap with the recent work [H. H. Bauschke, J. Bolte, and M. Teboulle, Math. Oper. Res., 42 (2017), pp. 330-348].
Date issued
2018-02Department
Massachusetts Institute of Technology. Department of Mathematics; Sloan School of ManagementJournal
SIAM Journal on Optimization
Publisher
Society for Industrial & Applied Mathematics (SIAM)
Citation
Lu, Haihao et al. “Relatively Smooth Convex Optimization by First-Order Methods, and Applications.” SIAM Journal on Optimization 28, no. 1 (January 2018): 333–354 © SIAM
Version: Final published version
ISSN
1052-6234
1095-7189