Acceleration in First Order Quasi-strongly Convex Optimization by ODE Discretization
Name
1905.12436.pdf
Description
Submitted version
Size
424.88 KB
Format
Adobe PDF
Checksum (MD5)
e155f6670dd2ba2247a98e2ceca08b4a
Author(s)
Zhang, Jingzhao
Sra, Suvrit
Jadbabaie, Ali
Date Issued
December 2019
Journal
Proceedings of the IEEE Conference on Decision and Control
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Zhang, Jingzhao et al. “Acceleration in First Order Quasi-strongly Convex Optimization by ODE Discretization.” Paper presented in the Proceedings of the IEEE Conference on Decision and Control, Nice, France , December 11-13 2019, Institute of Electrical and Electronics Engineers (IEEE) © 2019 The Author(s)
Version
Original manuscript
Abstract
We study gradient-based optimization methods obtained by direct Runge-Kutta discretization of the ordinary differential equation (ODE) describing the movement of a heavy-ball under constant friction coefficient. When the function is high-order smooth and strongly convex, we show that directly simulating the ODE with known numerical integrators achieve acceleration in a nontrivial neighborhood of the optimal solution. In particular, the neighborhood may grow larger as the condition number of the function increases. Furthermore, our results also hold for nonconvex but quasi-strongly convex objectives. We provide numerical experiments that verify the theoretical rates predicted by our results.
MIT Department
Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/CDC40024.2019.9030046
