Function approximation by deep networks
Name
1534-0392_2020_8_4085.pdf
Description
Published version
Size
514.57 KB
Format
Unknown
Checksum (MD5)
85f0d68d05338e7c0175648a8754eb7a
Author(s)
Mhaskar, H. N.
Poggio, Tomaso A
Date Issued
August 1, 2020
Journal
Communications on Pure and Applied Analysis
Publisher
American Institute of Mathematical Sciences (AIMS)
Citation
Mhaskar, HN and Poggio, T. 2020. "Function approximation by deep networks." Communications on Pure and Applied Analysis, 19 (8).
Version
Final published version
Abstract
© 2020 American Institute of Mathematical Sciences. All rights reserved. We show that deep networks are better than shallow networks at approximating functions that can be expressed as a composition of functions described by a directed acyclic graph, because the deep networks can be designed to have the same compositional structure, while a shallow network cannot exploit this knowledge. Thus, the blessing of compositionality mitigates the curse of dimensionality. On the other hand, a theorem called good propagation of errors allows to "lift" theorems about shallow networks to those about deep networks with an appropriate choice of norms, smoothness, etc. We illustrate this in three contexts where each channel in the deep network calculates a spherical polynomial, a non-smooth ReLU network, or another zonal function network related closely with the ReLU network.
MIT Department
Center for Brains, Minds, and Machines
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.3934/cpaa.2020181
