Learning Real and Boolean Functions: When Is Deep Better Than Shallow
Name
CBMM-Memo-045.pdf
Size
634.37 KB
Format
Adobe PDF
Checksum (MD5)
63dd0a7886bc66b5567c38108edb3aff
Author(s)
Mhaskar, Hrushikesh
Liao, Qianli
Poggio, Tomaso
Date Issued
March 8, 2016
Publisher
Center for Brains, Minds and Machines (CBMM), arXiv
Citation
arXiv:1603.00988
Series/Report no.
CBMM Memo Series;045
Abstract
We describe computational tasks - especially in vision - that correspond to compositional/hierarchical functions. While the universal approximation property holds both for hierarchical and shallow networks, we prove that deep (hierarchical) networks can approximate the class of compositional functions with the same accuracy as shallow networks but with exponentially lower VC-dimension as well as the number of training parameters. This leads to the question of approximation by sparse polynomials (in the number of independent parameters) and, as a consequence, by deep networks. We also discuss connections between our results and learnability of sparse Boolean functions, settling an old conjecture by Bengio.
Subjects
computational tasks
Computer vision
Hierarchy
Terms of Use
Attribution-NonCommercial-ShareAlike 3.0 United States
Persistent DSpace Link
