Unsupervised learning of invariant representations
Name
Poggio_Unsupervised learning.pdf
Size
4.25 MB
Format
Adobe PDF
Checksum (MD5)
0ad07fde7f82f327b69094825a5cc1c2
Author(s)
Anselmi, Fabio
Leibo, Joel Z
Rosasco, Lorenzo
Mutch, James Vincent
Tacchetti, Andrea
Poggio, Tomaso A
Date Issued
June 2015
Journal
Theoretical Computer Science
Publisher
Elsevier
Citation
Anselmi, Fabio, et al. “Unsupervised Learning of Invariant Representations.” Theoretical Computer Science, vol. 633, June 2016, pp. 112–21.
Version
Original manuscript
Abstract
The present phase of Machine Learning is characterized by supervised learning algorithms relying on large sets of labeled examples (n → ∞). The next phase is likely to focus on algorithms capable of learning from very few labeled examples (n → 1), like humans seem able to do. We propose an approach to this problem and describe the underlying theory, based on the unsupervised, automatic learning of a “good” representation for supervised learning, characterized by small sample complexity. We consider the case of visual object recognition, though the theory also applies to other domains like speech. The starting point is the conjecture, proved in specific cases, that image representations which are invariant to translation, scaling and other transformations can considerably reduce the sample complexity of learning. We prove that an invariant and selective signature can be computed for each image or image patch: the invariance can be exact in the case of group transformations and approximate under non-group transformations. A module performing filtering and pooling, like the simple and complex cells described by Hubel and Wiesel, can compute such signature. The theory offers novel unsupervised learning algorithms for “deep” architectures for image and speech recognition. We conjecture that the main computational goal of the ventral stream of visual cortex is to provide a hierarchical representation of new objects/images which is invariant to transformations, stable, and selective for recognition—and show how this representation may be continuously learned in an unsupervised way during development and visual experience.
MIT Department
Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences
McGovern Institute for Brain Research at MIT
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1016/j.tcs.2015.06.048
