Search
Now showing items 1-10 of 66
Streaming Normalization: Towards Simpler and More Biologically-plausible Normalizations for Online and Recurrent Learning
(Center for Brains, Minds and Machines (CBMM), arXiv, 2016-10-19)
We systematically explored a spectrum of normalization algorithms related to Batch Normalization (BN) and propose a generalized formulation that simultaneously solves two major limitations of BN: (1) online learning and ...
Foveation-based Mechanisms Alleviate Adversarial Examples
(Center for Brains, Minds and Machines (CBMM), arXiv, 2016-01-19)
We show that adversarial examples, i.e., the visually imperceptible perturbations that result in Convolutional Neural Networks (CNNs) fail, can be alleviated with a mechanism based on foveations---applying the CNN in ...
Theory IIIb: Generalization in Deep Networks
(Center for Brains, Minds and Machines (CBMM), arXiv.org, 2018-06-29)
The general features of the optimization problem for the case of overparametrized nonlinear networks have been clear for a while: SGD selects with high probability global minima vs local minima. In the overparametrized ...
Classical generalization bounds are surprisingly tight for Deep Networks
(Center for Brains, Minds and Machines (CBMM), 2018-07-11)
Deep networks are usually trained and tested in a regime in which the training classification error is not a good predictor of the test error. Thus the consensus has been that generalization, defined as convergence of the ...
Computational role of eccentricity dependent cortical magnification
(Center for Brains, Minds and Machines (CBMM), arXiv, 2014-06-06)
We develop a sampling extension of M-theory focused on invariance to scale and translation. Quite surprisingly, the theory predicts an architecture of early vision with increasing receptive field sizes and a high resolution ...
Fast, invariant representation for human action in the visual system
(Center for Brains, Minds and Machines (CBMM), arXiv, 2016-01-06)
The ability to recognize the actions of others from visual input is essential to humans' daily lives. The neural computations underlying action recognition, however, are still poorly understood. We use magnetoencephalography ...
Double descent in the condition number
(Center for Brains, Minds and Machines (CBMM), 2019-12-04)
In solving a system of n linear equations in d variables Ax=b, the condition number of the (n,d) matrix A measures how much errors in the data b affect the solution x. Bounds of this type are important in many inverse ...
Representation Learning in Sensory Cortex: a theory
(Center for Brains, Minds and Machines (CBMM), 2014-11-14)
We review and apply a computational theory of the feedforward path of the ventral stream in visual cortex based on the hypothesis that its main function is the encoding of invariant representations of images. A key ...
View-tolerant face recognition and Hebbian learning imply mirror-symmetric neural tuning to head orientation
(Center for Brains, Minds and Machines (CBMM), arXiv, 2016-06-03)
The primate brain contains a hierarchy of visual areas, dubbed the ventral stream, which rapidly computes object representations that are both specific for object identity and relatively robust against identity-preserving ...
Bridging the Gaps Between Residual Learning, Recurrent Neural Networks and Visual Cortex
(Center for Brains, Minds and Machines (CBMM), arXiv, 2016-04-12)
We discuss relations between Residual Networks (ResNet), Recurrent Neural Networks (RNNs) and the primate visual cortex. We begin with the observation that a shallow RNN is exactly equivalent to a very deep ResNet with ...