Size-independent sample complexity of neural networks
Name
1712.06541.pdf
Description
Accepted version
Size
310.54 KB
Format
Adobe PDF
Checksum (MD5)
399fbd6b2d00bd03b8846ca8debb6b51
Author(s)
Golowich, Noah
Rakhlin, Alexander
Shamir, Ohad
Date Issued
2020
Journal
Information and Inference
Publisher
Oxford University Press (OUP)
Citation
Golowich, Noah, Rakhlin, Alexander and Shamir, Ohad. 2020. "Size-independent sample complexity of neural networks." Information and Inference, 9 (2).
Version
Author's final manuscript
Abstract
We study the sample complexity of learning neural networks by providing new bounds on their Rademacher complexity, assuming norm constraints on the parameter matrix of each layer. Compared to previous work, these complexity bounds have improved dependence on the network depth and, under some additional assumptions, are fully independent of the network size (both depth and width). These results are derived using some novel techniques, which may be of independent interest.
MIT Department
Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences
Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
Massachusetts Institute of Technology. Institute for Data, Systems, and Society
Statistics and Data Science Center (Massachusetts Institute of Technology)
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1093/IMAIAI/IAZ007
