Show simple item record

dc.contributor.authorHuang, Shao-Lun
dc.contributor.authorXu, Xiangxiang
dc.contributor.authorZheng, Lizhong
dc.contributor.authorWornell, Gregory W
dc.date.accessioned2021-11-09T16:06:15Z
dc.date.available2021-11-09T16:06:15Z
dc.date.issued2019-07
dc.identifier.urihttps://hdl.handle.net/1721.1/137944
dc.description.abstract© 2019 IEEE. It is commonly believed that the hidden layers of deep neural networks (DNNs) attempt to extract informative features for learning tasks. In this paper, we formalize this intuition by showing that the features extracted by DNN coincide with the result of an optimization problem, which we call the "universal feature selection" problem, in a local analysis regime. We interpret the weights training in DNN as the projection of feature functions between feature spaces, specified by the network structure. Our formulation has direct operational meaning in terms of the performance for inference tasks, and gives interpretations to the internal computation results of DNNs. Results of numerical experiments are provided to support the analysis.en_US
dc.language.isoen
dc.publisherInstitute of Electrical and Electronics Engineers (IEEE)en_US
dc.relation.isversionof10.1109/ISIT.2019.8849720en_US
dc.rightsCreative Commons Attribution-Noncommercial-Share Alikeen_US
dc.rights.urihttp://creativecommons.org/licenses/by-nc-sa/4.0/en_US
dc.sourcearXiven_US
dc.titleAn Information Theoretic Interpretation to Deep Neural Networksen_US
dc.typeArticleen_US
dc.identifier.citationHuang, Shao-Lun, Xu, Xiangxiang, Zheng, Lizhong and Wornell, Gregory W. 2019. "An Information Theoretic Interpretation to Deep Neural Networks." IEEE International Symposium on Information Theory - Proceedings, 2019-July.
dc.relation.journalIEEE International Symposium on Information Theory - Proceedingsen_US
dc.eprint.versionOriginal manuscripten_US
dc.type.urihttp://purl.org/eprint/type/ConferencePaperen_US
eprint.statushttp://purl.org/eprint/status/NonPeerRevieweden_US
dc.date.updated2021-01-25T18:23:31Z
dspace.orderedauthorsHuang, S-L; Xu, X; Zheng, L; Wornell, GWen_US
dspace.date.submission2021-01-25T18:23:34Z
mit.journal.volume2019-Julyen_US
mit.licenseOPEN_ACCESS_POLICY
mit.metadata.statusAuthority Work and Publication Information Neededen_US


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record