Deep Convolutional Networks are Hierarchical Kernel Machines
Name
CBMM-Memo-035.pdf
Size
975.65 KB
Format
Adobe PDF
Checksum (MD5)
161dde7b68c0bf465585f881e3948dec
Author(s)
Anselmi, Fabio
Rosasco, Lorenzo
Tan, Cheston
Poggio, Tomaso
Date Issued
August 5, 2015
Publisher
Center for Brains, Minds and Machines (CBMM), arXiv
Citation
arXiv:1508.01084
Series/Report no.
CBMM Memo Series;035
Abstract
We extend i-theory to incorporate not only pooling but also rectifying nonlinearities in an extended HW module (eHW) designed for supervised learning. The two operations roughly correspond to invariance and selectivity, respectively. Under the assumption of normalized inputs, we show that appropriate linear combinations of rectifying nonlinearities are equivalent to radial kernels. If pooling is present an equivalent kernel also exist. Thus present-day DCNs (Deep Convolutional Networks) can be exactly equivalent to a hierarchy of kernel machines with pooling and non-pooling layers. Finally, we describe a conjecture for theoretically understanding hierarchies of such modules. A main consequence of the conjecture is that hierarchies of eHW modules minimize memory requirements while computing a selective and invariant representation.
Subjects
i-theory
extended HW module (eHW)
Invariance
Selectivity
Hierarchy
Machine Learning
Terms of Use
Attribution-NonCommercial 3.0 United States
Persistent DSpace Link
