dc.contributor.author | Liang, T | |
dc.contributor.author | Poggio, T | |
dc.contributor.author | Rakhlin, A | |
dc.contributor.author | Stokes, J | |
dc.date.accessioned | 2021-12-02T20:14:53Z | |
dc.date.available | 2021-12-02T20:14:53Z | |
dc.date.issued | 2020-01-01 | |
dc.identifier.uri | https://hdl.handle.net/1721.1/138296 | |
dc.description.abstract | © 2019 by the author(s). We study the relationship between geometry and capacity measures for deep neural networks from an invariance viewpoint. We introduce a new notion of capacity - the Fisher-Rao norm - that possesses desirable invariance properties and is motivated by Information Geometry. We discover an analytical characterization of the new capacity measure, through which we establish norm-comparison inequalities and further show that the new measure serves as an umbrella for several existing norm-based complexity measures. We discuss upper bounds on the generalization error induced by the proposed measure. Extensive numerical experiments on CIFAR-10 support our theoretical findings. Our theoretical analysis rests on a key structural lemma about partial derivatives of multi-layer rectifier networks. | en_US |
dc.language.iso | en | |
dc.relation.isversionof | http://proceedings.mlr.press/v89/liang19a/liang19a.pdf | en_US |
dc.rights | Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. | en_US |
dc.source | Proceedings of Machine Learning Research | en_US |
dc.title | Fisher-rao metric, geometry, and complexity of neural networks | en_US |
dc.type | Article | en_US |
dc.identifier.citation | Liang, T, Poggio, T, Rakhlin, A and Stokes, J. 2020. "Fisher-rao metric, geometry, and complexity of neural networks." AISTATS 2019 - 22nd International Conference on Artificial Intelligence and Statistics, 89. | |
dc.contributor.department | Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences | |
dc.contributor.department | McGovern Institute for Brain Research at MIT | |
dc.contributor.department | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory | |
dc.contributor.department | Statistics and Data Science Center (Massachusetts Institute of Technology) | |
dc.contributor.department | Massachusetts Institute of Technology. Institute for Data, Systems, and Society | |
dc.contributor.department | Massachusetts Institute of Technology. Laboratory for Information and Decision Systems | |
dc.relation.journal | AISTATS 2019 - 22nd International Conference on Artificial Intelligence and Statistics | en_US |
dc.eprint.version | Final published version | en_US |
dc.type.uri | http://purl.org/eprint/type/ConferencePaper | en_US |
eprint.status | http://purl.org/eprint/status/NonPeerReviewed | en_US |
dc.date.updated | 2021-12-02T20:10:01Z | |
dspace.orderedauthors | Liang, T; Poggio, T; Rakhlin, A; Stokes, J | en_US |
dspace.date.submission | 2021-12-02T20:10:02Z | |
mit.journal.volume | 89 | en_US |
mit.license | PUBLISHER_POLICY | |
mit.metadata.status | Authority Work and Publication Information Needed | en_US |