Relatively Smooth Convex Optimization by First-Order Methods, and Applications
Name
16m1099546.pdf
Size
280.38 KB
Format
Adobe PDF
Checksum (MD5)
a6aafe3e9973cfc880b453ef21f571bc
Author(s)
Nesterov, Yurii
Lu, Haihao
Freund, Robert Michael
Date Issued
February 2018
Journal
SIAM Journal on Optimization
Publisher
Society for Industrial & Applied Mathematics (SIAM)
Citation
Lu, Haihao et al. “Relatively Smooth Convex Optimization by First-Order Methods, and Applications.” SIAM Journal on Optimization 28, no. 1 (January 2018): 333–354 © SIAM
Version
Final published version
Abstract
The usual approach to developing and analyzing first-order methods for smooth convex optimization assumes that the gradient of the objective function is uniformly smooth with some Lipschitz constant L. However, in many settings the differentiable convex function f(?) is not uniformly smooth-for example, in D-optimal design where f(x) := -ln det(HXHT) and X := Diag(x), or even the univariate setting with f(x) := -ln(x)+x2. In this paper we develop a notion of "relative smoothness" and relative strong convexity that is determined relative to a user-specified "reference function" h(?) (that should be computationally tractable for algorithms), and we show that many differentiable convex functions are relatively smooth with respect to a correspondingly fairly simple reference function h(?). We extend two standard algorithms-the primal gradient scheme and the dual averaging scheme-to our new setting, with associated computational guarantees. We apply our new approach to develop a new first-order method for the D-optimal design problem, with associated computational complexity analysis. Some of our results have a certain overlap with the recent work [H. H. Bauschke, J. Bolte, and M. Teboulle, Math. Oper. Res., 42 (2017), pp. 330-348].
MIT Department
Massachusetts Institute of Technology. Department of Mathematics
Sloan School of Management
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1137/16M1099546
