2026-09-19T19:12:20Zhttps://dspace.mit.edu/server/oai/request

oai:dspace.mit.edu:1721.1/1308592021-07-05T14:03:20Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023

Anuradha Annaswamy. Moreu Gamazo, José M.(José María) Massachusetts Institute of Technology. Department of Mechanical Engineering. Massachusetts Institute of Technology. Department of Mechanical Engineering 2021-05-25T18:23:26Z 2021-05-25T18:23:26Z 2021 2021 https://hdl.handle.net/1721.1/130859 1252630722 Thesis: S.M., Massachusetts Institute of Technology, Department of Mechanical Engineering, February, 2021 Cataloged from the official PDF version of thesis. Includes bibliographical references (pages 131-132). Iterative gradient-based algorithms have been increasingly applied for the training of a broad variety of machine learning models including large neural-nets. In particular, momentum-based methods, with accelerated learning guarantees, have received a lot of attention due to their provable guarantees of fast learning in certain classes of problems and multiple algorithms have been derived. However, properties for these methods hold true only for constant regressors. When time-varying regressors occur, which is commonplace in dynamic systems, many of these momentum-based methods cannot guarantee stability. Recently, a new High-order Tuner (HT) was developed and shown to have 1) stability and asymptotic convergence for time-varying regressors and 2) non-asymptotic accelerated learning guarantees for constant regressors. These results were derived for a linear regression framework producing a quadratic loss function. This thesis extends and discuss the results of this same HT for general smooth convex loss functions. Through the exploitation of convexity and smoothness definitions, we establish similar stability and asymptotic convergence guarantees. Additionally we conjecture that the HT has an accelerated convergence rate. Finally, we provide numerical simulations supporting the satisfactory behavior of the HT algorithm as well as the conjecture of accelerated learning. by José M. Moreu Gamazo. S.M. S.M. Massachusetts Institute of Technology, Department of Mechanical Engineering 132 pages eng Massachusetts Institute of Technology MIT theses may be protected by copyright. Please reuse MIT thesis content according to the MIT Libraries Permissions Policy, which is available through the URL provided. http://dspace.mit.edu/handle/1721.1/7582 Mechanical Engineering. High-order tuners for convex optimization : stability and accelerated learning Thesis application/pdf 2021-05-25T18:23:26Z Publication Master MechE unknown unknown <Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="a54a4507-869e-4e92-ae2c-b99e727879b1"> <Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type> <Language>eng</Language> <Title>High-order tuners for convex optimization : stability and accelerated learning</Title> <PublishedIn> <Publication> </Publication> </PublishedIn> <PublicationDate>2021</PublicationDate> <Authors> <Author> <DisplayName>Moreu Gamazo, José M.(José María)</DisplayName> <Affiliation> <OrgUnit> </OrgUnit> </Affiliation> </Author> </Authors> <Editors> </Editors> <Publishers> <Publisher> <DisplayName>Massachusetts Institute of Technology</DisplayName> <OrgUnit /> </Publisher> </Publishers> <License>http://dspace.mit.edu/handle/1721.1/7582</License> <Keyword>Mechanical Engineering.</Keyword> <Abstract>Iterative gradient-based algorithms have been increasingly applied for the training of a broad variety of machine learning models including large neural-nets. In particular, momentum-based methods, with accelerated learning guarantees, have received a lot of attention due to their provable guarantees of fast learning in certain classes of problems and multiple algorithms have been derived. However, properties for these methods hold true only for constant regressors. When time-varying regressors occur, which is commonplace in dynamic systems, many of these momentum-based methods cannot guarantee stability. Recently, a new High-order Tuner (HT) was developed and shown to have 1) stability and asymptotic convergence for time-varying regressors and 2) non-asymptotic accelerated learning guarantees for constant regressors. These results were derived for a linear regression framework producing a quadratic loss function. This thesis extends and discuss the results of this same HT for general smooth convex loss functions. Through the exploitation of convexity and smoothness definitions, we establish similar stability and asymptotic convergence guarantees. Additionally we conjecture that the HT has an accelerated convergence rate. Finally, we provide numerical simulations supporting the satisfactory behavior of the HT algorithm as well as the conjecture of accelerated learning.</Abstract> <Access xmlns="http://purl.org/coar/access_right" > </Access> </Publication>