2026-09-19T00:36:50Zhttps://dspace.mit.edu/server/oai/requestoai:dspace.mit.edu:1721.1/1089732022-01-13T07:54:01Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023
Constantinos Daskalakis.
Zampetakis, Emmanouil
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
2017-05-11T19:58:35Z
2017-05-11T19:58:35Z
2017
2017
http://hdl.handle.net/1721.1/108973
986497232
Thesis: S.M., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2017.
Cataloged from PDF version of thesis.
Includes bibliographical references (pages 101-107).
The increasing interest of the scientific community, and especially machine learning, on non-convex problems, has made non-convex optimization one of the most important and challenging areas of our days. Despite of this increasing interest too little is known from a theoretical point of view. The main reason for this is that the existing and well understood techniques used for the analysis of convex optimization problem are not applicable or meaningful in the non-convex case. The purpose of this thesis is to make a step in the direction of investigating a rich enough toolbox, to be able to analyze non-convex optimization. Contraction maps and Banach's Fixed Point Theorem are very important tools for bounding the running time of a big class of iterative algorithms used to solve non-convex problems. But when we use the natural distance metric, of the spaces that we are working on, the applicability of Banach's Fixed Point Theorem becomes limited. The reason is that only few functions have the contraction property with the natural metrics. We explore how generally we can apply Banach's fixed point theorem to establish the convergence of iterative methods when pairing it with carefully designed metrics. Our first result is a strong converse of Banach's theorem, showing that it is a universal analysis tool for establishing uniqueness of fixed points and convergence of iterative maps to a unique solution. We next consider the computational complexity of Banach's fixed point theorem. Making the proof of our converse theorem constructive, we show that computing Banach's fixed point theorem is CLS-complete, answering a question left open in the work of Daskalakis and Papadimitriou [23]. Finally, we turn to applications proving global convergence guarantees for one of the most celebrated inference algorithms in Statistics, the EM algorithm. Proposed in the 70's [26], the EM algorithm is an iterative method for maximum likelihood estimation whose behavior has vastly remained elusive. We show that it converges to the true optimum for balanced mixtures of two Gaussians.
by Emmanouil Zampetakis.
S.M.
107 pages
eng
Massachusetts Institute of Technology
MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission.
http://dspace.mit.edu/handle/1721.1/7582
Electrical Engineering and Computer Science.
Contraction maps and applications to the analysis of iterative algorithms
Thesis
application/pdf
false
Publication
unknown
unknown
<Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="413c1e64-9daa-4b71-b85e-c89ec6cfd12c">
<Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type>
<Language>eng</Language>
<Title>Contraction maps and applications to the analysis of iterative algorithms</Title>
<PublishedIn>
<Publication>
</Publication>
</PublishedIn>
<PublicationDate>2017</PublicationDate>
<Authors>
<Author>
<DisplayName>Zampetakis, Emmanouil</DisplayName>
<Affiliation>
<OrgUnit>
</OrgUnit>
</Affiliation>
</Author>
</Authors>
<Editors>
</Editors>
<Publishers>
<Publisher>
<DisplayName>Massachusetts Institute of Technology</DisplayName>
<OrgUnit />
</Publisher>
</Publishers>
<License>http://dspace.mit.edu/handle/1721.1/7582</License>
<Keyword>Electrical Engineering and Computer Science.</Keyword>
<Abstract>The increasing interest of the scientific community, and especially machine learning, on non-convex problems, has made non-convex optimization one of the most important and challenging areas of our days. Despite of this increasing interest too little is known from a theoretical point of view. The main reason for this is that the existing and well understood techniques used for the analysis of convex optimization problem are not applicable or meaningful in the non-convex case. The purpose of this thesis is to make a step in the direction of investigating a rich enough toolbox, to be able to analyze non-convex optimization. Contraction maps and Banach's Fixed Point Theorem are very important tools for bounding the running time of a big class of iterative algorithms used to solve non-convex problems. But when we use the natural distance metric, of the spaces that we are working on, the applicability of Banach's Fixed Point Theorem becomes limited. The reason is that only few functions have the contraction property with the natural metrics. We explore how generally we can apply Banach's fixed point theorem to establish the convergence of iterative methods when pairing it with carefully designed metrics. Our first result is a strong converse of Banach's theorem, showing that it is a universal analysis tool for establishing uniqueness of fixed points and convergence of iterative maps to a unique solution. We next consider the computational complexity of Banach's fixed point theorem. Making the proof of our converse theorem constructive, we show that computing Banach's fixed point theorem is CLS-complete, answering a question left open in the work of Daskalakis and Papadimitriou [23]. Finally, we turn to applications proving global convergence guarantees for one of the most celebrated inference algorithms in Statistics, the EM algorithm. Proposed in the 70's [26], the EM algorithm is an iterative method for maximum likelihood estimation whose behavior has vastly remained elusive. We show that it converges to the true optimum for balanced mixtures of two Gaussians.</Abstract>
<Access xmlns="http://purl.org/coar/access_right"
>
</Access>
</Publication>