Theses - Dept. of Mathematicshttps://hdl.handle.net/1721.1/76042019-06-25T14:23:17Z2019-06-25T14:23:17ZReliable validation : new perspectives on adaptive data analysis and cross-validationElder, Samuel Scotthttps://hdl.handle.net/1721.1/1206602019-04-10T15:37:00Z2018-01-01T00:00:00ZReliable validation : new perspectives on adaptive data analysis and cross-validation
Elder, Samuel Scott
Validation refers to the challenge of assessing how well a learning algorithm performs after it has been trained on a given data set. It forms an important step in machine learning, as such assessments are then used to compare and choose between algorithms and provide reasonable approximations of their accuracy. In this thesis, we provide new approaches for addressing two common problems with validation. In the first half, we assume a simple validation framework, the holdout set, and address an important question of how many algorithms can be accurately assessed using the same holdout set, in the particular case where these algorithms are chosen adaptively. We do so by first critiquing the initial approaches to building a theory of adaptivity, then offering an alternative approach and preliminary results within this approach, all geared towards characterizing the inherent challenge of adaptivity. In the second half, we address the validation framework itself. Most common practice does not just use a single holdout set, but averages results from several, a family of techniques known as cross-validation. In this work, we offer several new cross-validation techniques with the common theme of utilizing training sets of varying sizes. This culminates in hierarchical cross-validation, a meta-technique for using cross-validation to choose the best cross-validation method.
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Mathematics, 2018.; Cataloged from PDF version of thesis.; Includes bibliographical references (pages 107-109).
2018-01-01T00:00:00ZStatistical limits of graphical channel models and a semidefinite programming approachKim, Chiheonhttps://hdl.handle.net/1721.1/1206592019-04-10T22:54:26Z2018-01-01T00:00:00ZStatistical limits of graphical channel models and a semidefinite programming approach
Kim, Chiheon
Community recovery is a major challenge in data science and computer science. The goal in community recovery is to find the hidden clusters from given relational data, which is often represented as a labeled hyper graph where nodes correspond to items needing to be labeled and edges correspond to observed relations between the items. We investigate the problem of exact recovery in the class of statistical models which can be expressed in terms of graphical channels. In a graphical channel model, we observe noisy measurements of the relations between k nodes while the true labeling is unknown to us, and the goal is to recover the labels correctly. This generalizes both the stochastic block models and spiked tensor models for principal component analysis, which has gained much interest over the last decade. We focus on two aspects of exact recovery: statistical limits and efficient algorithms achieving the statistic limit. For the statistical limits, we show that the achievability of exact recovery is essentially determined by whether we can recover the label of one node given other nodes labels with fairly high probability. This phenomenon was observed by Abbe et al. for generic stochastic block models, and called "local-to-global amplification". We confirm that local-to-global amplification indeed holds for generic graphical channel models, under some regularity assumptions. As a corollary, the threshold for exact recovery is explicitly determined. For algorithmic concerns, we consider two examples of graphical channel models, (i) the spiked tensor model with additive Gaussian noise, and (ii) the generalization of the stochastic block model for k-uniform hypergraphs. We propose a strategy which we call "truncate-and-relax", based on a standard semidefinite relaxation technique. We show that in these two models, the algorithm based on this strategy achieves exact recovery up to a threshold which orderwise matches the statistical threshold. We complement this by showing the limitation of the algorithm.
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Mathematics, 2018.; Cataloged from PDF version of thesis.; Includes bibliographical references (pages 205-213).
2018-01-01T00:00:00ZTowards an integrated understanding of neural networksRolnick, David (David S.)https://hdl.handle.net/1721.1/1206582019-04-10T19:59:01Z2018-01-01T00:00:00ZTowards an integrated understanding of neural networks
Rolnick, David (David S.)
Neural networks underpin both biological intelligence and modern Al systems, yet there is relatively little theory for how the observed behavior of these networks arises. Even the connectivity of neurons within the brain remains largely unknown, and popular deep learning algorithms lack theoretical justification or reliability guarantees. This thesis aims towards a more rigorous understanding of neural networks. We characterize and, where possible, prove essential properties of neural algorithms: expressivity, learning, and robustness. We show how observed emergent behavior can arise from network dynamics, and we develop algorithms for learning more about the network structure of the brain.
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Mathematics, 2018.; Cataloged from PDF version of thesis.; Includes bibliographical references (pages 123-136).
2018-01-01T00:00:00ZApplied math in geophysical fluids : partially trapped wave problems and mining plumesRzeznik, Andrew Josephhttps://hdl.handle.net/1721.1/1206572019-04-11T06:13:53Z2018-01-01T00:00:00ZApplied math in geophysical fluids : partially trapped wave problems and mining plumes
Rzeznik, Andrew Joseph
The first portion of this work focuses on leaky modes in the atmospheric sciences. Leaky modes (related to quasi-modes, scattering resonances, and the singularity expansion method) are discrete, oscillatory and decaying modes that arise in conservative systems where waves are partially trapped. By replacing the infinite domain with a finite domain and appropriate boundary conditions it is possible in many cases to construct a complete basis for the solution in terms of these modes. Formulating such effective boundary conditions requires a notion of the direction of propagation of the waves. For this purpose we introduce a generalization of the concept of group speed for exponentially decaying but conservative waves. This is found via an extended modulation argument and a generalization of Whitham's Average Lagrangian theory. The theory also shows that a close relationship exists between the branch cuts of the dispersion relation and the propagation direction, and is used to create spectral decompositions for simple problems in internal gravity waves. The last chapter considers deep-sea nodule mining operations, which potentially involve plans for discharge plumes to be released into the water column by surface operation vessels. We consider the effects of non-uniform, realistic stratifications with vertical shear on forced compressible plumes. The plume model is developed to account for the influence of thermal conduction through the discharge pipe and an initial adjustment phase. We investigate the substantial role of compressibility, for which a dimensionless number is introduced to determine its importance compared to that of the background stratification. Our results show that (i) small-scale stratification features can have a significant impact, (ii) in a static ambient there exists a discharge flow rate that minimizes the plume vertical extent, (iii) the ambient velocity profile plays an important role in determining the final plume scale and dilution factor, and (iv) for a typical plume the dilution factor is expected to be several hundred to a thousand.
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Mathematics, 2018.; Cataloged from PDF version of thesis.; Includes bibliographical references (pages 129-132).
2018-01-01T00:00:00Z