Search
Now showing items 1-6 of 6
Theory IIIb: Generalization in Deep Networks
(Center for Brains, Minds and Machines (CBMM), arXiv.org, 2018-06-29)
The general features of the optimization problem for the case of overparametrized nonlinear networks have been clear for a while: SGD selects with high probability global minima vs local minima. In the overparametrized ...
Classical generalization bounds are surprisingly tight for Deep Networks
(Center for Brains, Minds and Machines (CBMM), 2018-07-11)
Deep networks are usually trained and tested in a regime in which the training classification error is not a good predictor of the test error. Thus the consensus has been that generalization, defined as convergence of the ...
An analysis of training and generalization errors in shallow and deep networks
(Center for Brains, Minds and Machines (CBMM), arXiv.org, 2018-02-20)
An open problem around deep networks is the apparent absence of over-fitting despite large over-parametrization which allows perfect fitting of the training data. In this paper, we explain this phenomenon when each unit ...
Single units in a deep neural network functionally correspond with neurons in the brain: preliminary results
(Center for Brains, Minds and Machines (CBMM), 2018-11-02)
Deep neural networks have been shown to predict neural responses in higher visual cortex. The mapping from the model to a neuron in the brain occurs through a linear combination of many units in the model, leaving open the ...
Biologically-Plausible Learning Algorithms Can Scale to Large Datasets
(Center for Brains, Minds and Machines (CBMM), 2018-09-27)
The backpropagation (BP) algorithm is often thought to be biologically implausible in the brain. One of the main reasons is that BP requires symmetric weight matrices in the feedforward and feed- back pathways. To address ...
Biologically-plausible learning algorithms can scale to large datasets
(Center for Brains, Minds and Machines (CBMM), arXiv.org, 2018-11-08)
The backpropagation (BP) algorithm is often thought to be biologically implausible in the brain. One of the main reasons is that BP requires symmetric weight matrices in the feedforward and feedback pathways. To address ...