Theses - Dept. of Mathematics
https://hdl.handle.net/1721.1/7604
2019-07-24T08:44:19ZProblems in discrete applied mathematics
https://hdl.handle.net/1721.1/121905
Problems in discrete applied mathematics
Assmann, Susan Fera.
Thesis (Ph.D.)--Massachusetts Institute of Technology, Dept. of Mathematics, 1983.; Vita.; Bibliography: leaves 120-124.
1983-01-01T00:00:00ZReliable validation : new perspectives on adaptive data analysis and cross-validation
https://hdl.handle.net/1721.1/120660
Reliable validation : new perspectives on adaptive data analysis and cross-validation
Elder, Samuel Scott
Validation refers to the challenge of assessing how well a learning algorithm performs after it has been trained on a given data set. It forms an important step in machine learning, as such assessments are then used to compare and choose between algorithms and provide reasonable approximations of their accuracy. In this thesis, we provide new approaches for addressing two common problems with validation. In the first half, we assume a simple validation framework, the holdout set, and address an important question of how many algorithms can be accurately assessed using the same holdout set, in the particular case where these algorithms are chosen adaptively. We do so by first critiquing the initial approaches to building a theory of adaptivity, then offering an alternative approach and preliminary results within this approach, all geared towards characterizing the inherent challenge of adaptivity. In the second half, we address the validation framework itself. Most common practice does not just use a single holdout set, but averages results from several, a family of techniques known as cross-validation. In this work, we offer several new cross-validation techniques with the common theme of utilizing training sets of varying sizes. This culminates in hierarchical cross-validation, a meta-technique for using cross-validation to choose the best cross-validation method.
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Mathematics, 2018.; Cataloged from PDF version of thesis.; Includes bibliographical references (pages 107-109).
2018-01-01T00:00:00ZStatistical limits of graphical channel models and a semidefinite programming approach
https://hdl.handle.net/1721.1/120659
Statistical limits of graphical channel models and a semidefinite programming approach
Kim, Chiheon
Community recovery is a major challenge in data science and computer science. The goal in community recovery is to find the hidden clusters from given relational data, which is often represented as a labeled hyper graph where nodes correspond to items needing to be labeled and edges correspond to observed relations between the items. We investigate the problem of exact recovery in the class of statistical models which can be expressed in terms of graphical channels. In a graphical channel model, we observe noisy measurements of the relations between k nodes while the true labeling is unknown to us, and the goal is to recover the labels correctly. This generalizes both the stochastic block models and spiked tensor models for principal component analysis, which has gained much interest over the last decade. We focus on two aspects of exact recovery: statistical limits and efficient algorithms achieving the statistic limit. For the statistical limits, we show that the achievability of exact recovery is essentially determined by whether we can recover the label of one node given other nodes labels with fairly high probability. This phenomenon was observed by Abbe et al. for generic stochastic block models, and called "local-to-global amplification". We confirm that local-to-global amplification indeed holds for generic graphical channel models, under some regularity assumptions. As a corollary, the threshold for exact recovery is explicitly determined. For algorithmic concerns, we consider two examples of graphical channel models, (i) the spiked tensor model with additive Gaussian noise, and (ii) the generalization of the stochastic block model for k-uniform hypergraphs. We propose a strategy which we call "truncate-and-relax", based on a standard semidefinite relaxation technique. We show that in these two models, the algorithm based on this strategy achieves exact recovery up to a threshold which orderwise matches the statistical threshold. We complement this by showing the limitation of the algorithm.
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Mathematics, 2018.; Cataloged from PDF version of thesis.; Includes bibliographical references (pages 205-213).
2018-01-01T00:00:00ZTowards an integrated understanding of neural networks
https://hdl.handle.net/1721.1/120658
Towards an integrated understanding of neural networks
Rolnick, David (David S.)
Neural networks underpin both biological intelligence and modern Al systems, yet there is relatively little theory for how the observed behavior of these networks arises. Even the connectivity of neurons within the brain remains largely unknown, and popular deep learning algorithms lack theoretical justification or reliability guarantees. This thesis aims towards a more rigorous understanding of neural networks. We characterize and, where possible, prove essential properties of neural algorithms: expressivity, learning, and robustness. We show how observed emergent behavior can arise from network dynamics, and we develop algorithms for learning more about the network structure of the brain.
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Mathematics, 2018.; Cataloged from PDF version of thesis.; Includes bibliographical references (pages 123-136).
2018-01-01T00:00:00Z