Testing, Learning, and Optimization in High Dimensions
Name
Gatmiry-gatmiry-SM-EECS-2022-thesis.pdf
Description
Thesis PDF
Size
1.65 MB
Format
Adobe PDF
Checksum (MD5)
a0609479f5ea0c64f3ffa24b059be319
Author(s)
Gatmiry, Khashayar
Advisor(s)
Stefanie Jegelka
Kelner, Jonathan
Date Issued
May 2022
Publisher
Massachusetts Institute of Technology
Abstract
In this thesis we study two separate problems: (1) What is the sample complexity of testing the class of Determinantal Point Processes? and (2) Introducing a new analysis for optimization and generalization of deep neural networks beyond their linear approximation. For the first problem, we characterize the optimal sample complexity up to logarithmic factors by proposing almost matching upper and lower bounds. For the second problem, we propose a new regime for the parameters and the algorithm of a three layer network model which goes beyond the Neural tangent kernel (NTK) approximation; as a result, we introduce a new data dependent complexity measure which generalizes the NTK complexity measure introduced by [Arora et al., 2019a]. We show that despite nonconvexity, a variant of Stochastic gradient descent (SGD) converges to a good solution for which we prove a novel generalization bound that is proportional to our complexity measure.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright MIT
Persistent DSpace Link
