Hierarchically Local Tasks and Deep Convolutional Networks
Name
CBMM_Memo_109.pdf
Size
2.12 MB
Format
Adobe PDF
Checksum (MD5)
888f5a4fc46d1aa525be364c8323d5de
Author(s)
Deza, Arturo
Liao, Qianli
Banburski, Andrzej
Poggio, Tomaso
Date Issued
June 24, 2020
Publisher
Center for Brains, Minds and Machines (CBMM)
Series/Report no.
CBMM Memo;109
Abstract
The main success stories of deep learning, starting with ImageNet, depend on convolutional networks, which on certain tasks perform significantly better than traditional shallow classifiers, such as support vector machines. Is there something special about deep convolutional networks that other learning machines do not possess? Recent results in approximation theory have shown that there is an exponential advantage of deep convolutional-like networks in approximating functions with hierarchical locality in their compositional structure. These mathematical results, however, do not say which tasks are expected to have input-output functions with hierarchical locality. Among all the possible hierarchically local tasks in vision, text and speech we explore a few of them experimentally by studying how they are affected by disrupting locality in the input images. We also discuss a taxonomy of tasks ranging from local, to hierarchically local, to global and make predictions about the type of networks required to perform efficiently on these different types of tasks.
Subjects
Compositionality
Inductive Bias
perception
Theory of Deep Learning
Persistent DSpace Link
