Show simple item record

dc.contributor.authorWai, Hoi-To
dc.contributor.authorShi, Wei
dc.contributor.authorUribe, César A
dc.contributor.authorNedić, Angelia
dc.contributor.authorScaglione, Anna
dc.date.accessioned2021-09-20T17:30:42Z
dc.date.available2021-09-20T17:30:42Z
dc.date.issued2020-03-07
dc.identifier.urihttps://hdl.handle.net/1721.1/131862
dc.description.abstractAbstract This paper studies an acceleration technique for incremental aggregated gradient (IAG) method through the use of curvature information for solving strongly convex finite sum optimization problems. These optimization problems of interest arise in large-scale learning applications. Our technique utilizes a curvature-aided gradient tracking step to produce accurate gradient estimates incrementally using Hessian information. We propose and analyze two methods utilizing the new technique, the curvature-aided IAG (CIAG) method and the accelerated CIAG (A-CIAG) method, which are analogous to gradient method and Nesterov’s accelerated gradient method, respectively. Setting $$\kappa$$κ to be the condition number of the objective function, we prove the R linear convergence rates of $$1 - \frac{4c_0 \kappa }{(\kappa +1)^2}$$1-4c0κ(κ+1)2 for the CIAG method, and $$1 - \sqrt{\frac{c_1}{2\kappa }}$$1-c12κ for the A-CIAG method, where $$c_0,c_1 \le 1$$c0,c1≤1 are constants inversely proportional to the distance between the initial point and the optimal solution. When the initial iterate is close to the optimal solution, the R linear convergence rates match with the gradient and accelerated gradient method, albeit CIAG and A-CIAG operate in an incremental setting with strictly lower computation complexity. Numerical experiments confirm our findings. The source codes used for this paper can be found on http://github.com/hoitowai/ciag/.en_US
dc.publisherSpringer USen_US
dc.relation.isversionofhttps://doi.org/10.1007/s10589-020-00183-1en_US
dc.rightsArticle is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.en_US
dc.sourceSpringer USen_US
dc.titleAccelerating incremental gradient optimization with curvature informationen_US
dc.typeArticleen_US
dc.contributor.departmentMassachusetts Institute of Technology. Laboratory for Information and Decision Systems
dc.eprint.versionAuthor's final manuscripten_US
dc.type.urihttp://purl.org/eprint/type/JournalArticleen_US
eprint.statushttp://purl.org/eprint/status/PeerRevieweden_US
dc.date.updated2020-09-24T21:34:35Z
dc.language.rfc3066en
dc.rights.holderSpringer Science+Business Media, LLC, part of Springer Nature
dspace.embargo.termsY
dspace.date.submission2020-09-24T21:34:35Z
mit.licensePUBLISHER_POLICY
mit.metadata.statusAuthority Work and Publication Information Needed


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record