Contraction maps and applications to the analysis of iterative algorithms
Name
986497232-MIT.pdf
Description
Full printable version
Size
5 MB
Format
Adobe PDF
Checksum (MD5)
69e30ed7c9b5214acd0b38a91aee3591
Author(s)
Zampetakis, Emmanouil
Advisor(s)
Constantinos Daskalakis.
Date Issued
2017
Publisher
Massachusetts Institute of Technology
Abstract
The increasing interest of the scientific community, and especially machine learning, on non-convex problems, has made non-convex optimization one of the most important and challenging areas of our days. Despite of this increasing interest too little is known from a theoretical point of view. The main reason for this is that the existing and well understood techniques used for the analysis of convex optimization problem are not applicable or meaningful in the non-convex case. The purpose of this thesis is to make a step in the direction of investigating a rich enough toolbox, to be able to analyze non-convex optimization. Contraction maps and Banach's Fixed Point Theorem are very important tools for bounding the running time of a big class of iterative algorithms used to solve non-convex problems. But when we use the natural distance metric, of the spaces that we are working on, the applicability of Banach's Fixed Point Theorem becomes limited. The reason is that only few functions have the contraction property with the natural metrics. We explore how generally we can apply Banach's fixed point theorem to establish the convergence of iterative methods when pairing it with carefully designed metrics. Our first result is a strong converse of Banach's theorem, showing that it is a universal analysis tool for establishing uniqueness of fixed points and convergence of iterative maps to a unique solution. We next consider the computational complexity of Banach's fixed point theorem. Making the proof of our converse theorem constructive, we show that computing Banach's fixed point theorem is CLS-complete, answering a question left open in the work of Daskalakis and Papadimitriou [23]. Finally, we turn to applications proving global convergence guarantees for one of the most celebrated inference algorithms in Statistics, the EM algorithm. Proposed in the 70's [26], the EM algorithm is an iterative method for maximum likelihood estimation whose behavior has vastly remained elusive. We show that it converges to the true optimum for balanced mixtures of two Gaussians.
Description
Thesis: S.M., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2017.
Cataloged from PDF version of thesis.
Includes bibliographical references (pages 101-107).
Subjects
Electrical Engineering and Computer Science.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission.
Persistent DSpace Link
