Show simple item record

dc.contributor.authorLiang, T
dc.contributor.authorPoggio, T
dc.contributor.authorRakhlin, A
dc.contributor.authorStokes, J
dc.date.accessioned2021-12-02T20:14:53Z
dc.date.available2021-12-02T20:14:53Z
dc.date.issued2020-01-01
dc.identifier.urihttps://hdl.handle.net/1721.1/138296
dc.description.abstract© 2019 by the author(s). We study the relationship between geometry and capacity measures for deep neural networks from an invariance viewpoint. We introduce a new notion of capacity - the Fisher-Rao norm - that possesses desirable invariance properties and is motivated by Information Geometry. We discover an analytical characterization of the new capacity measure, through which we establish norm-comparison inequalities and further show that the new measure serves as an umbrella for several existing norm-based complexity measures. We discuss upper bounds on the generalization error induced by the proposed measure. Extensive numerical experiments on CIFAR-10 support our theoretical findings. Our theoretical analysis rests on a key structural lemma about partial derivatives of multi-layer rectifier networks.en_US
dc.language.isoen
dc.relation.isversionofhttp://proceedings.mlr.press/v89/liang19a/liang19a.pdfen_US
dc.rightsArticle is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.en_US
dc.sourceProceedings of Machine Learning Researchen_US
dc.titleFisher-rao metric, geometry, and complexity of neural networksen_US
dc.typeArticleen_US
dc.identifier.citationLiang, T, Poggio, T, Rakhlin, A and Stokes, J. 2020. "Fisher-rao metric, geometry, and complexity of neural networks." AISTATS 2019 - 22nd International Conference on Artificial Intelligence and Statistics, 89.
dc.contributor.departmentMassachusetts Institute of Technology. Department of Brain and Cognitive Sciences
dc.contributor.departmentMcGovern Institute for Brain Research at MIT
dc.contributor.departmentMassachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
dc.contributor.departmentStatistics and Data Science Center (Massachusetts Institute of Technology)
dc.contributor.departmentMassachusetts Institute of Technology. Institute for Data, Systems, and Society
dc.contributor.departmentMassachusetts Institute of Technology. Laboratory for Information and Decision Systems
dc.relation.journalAISTATS 2019 - 22nd International Conference on Artificial Intelligence and Statisticsen_US
dc.eprint.versionFinal published versionen_US
dc.type.urihttp://purl.org/eprint/type/ConferencePaperen_US
eprint.statushttp://purl.org/eprint/status/NonPeerRevieweden_US
dc.date.updated2021-12-02T20:10:01Z
dspace.orderedauthorsLiang, T; Poggio, T; Rakhlin, A; Stokes, Jen_US
dspace.date.submission2021-12-02T20:10:02Z
mit.journal.volume89en_US
mit.licensePUBLISHER_POLICY
mit.metadata.statusAuthority Work and Publication Information Neededen_US


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record