CBMM Memo Series: Recent submissions

Now showing items 10-12 of 146

    • Feature learning in deep classifiers through Intermediate Neural Collapse 

      Rangamani, Akshay; Lindegaard, Marius; Galanti, Tomer; Poggio, Tomaso (Center for Brains, Minds and Machines (CBMM), 2023-02-27)
      In this paper, we conduct an empirical study of the feature learning process in deep classifiers. Recent research has identified a training phenomenon called Neural Collapse (NC), in which the top-layer feature embeddings ...

    • SGD and Weight Decay Provably Induce a Low-Rank Bias in Deep Neural Networks 

      Galanti, Tomer; Siegel, Zachary; Gupte, Aparna; Poggio, Tomaso (Center for Brains, Minds and Machines (CBMM), 2023-02-14)
      In this paper, we study the bias of Stochastic Gradient Descent (SGD) to learn low-rank weight matrices when training deep ReLU neural networks. Our results show that training neural networks with mini-batch SGD and weight ...

    • Norm-Based Generalization Bounds for Compositionally Sparse Neural Network 

      Galanti, Tomer; Xu, Mengjia; Galanti, Liane; Poggio, Tomaso (Center for Brains, Minds and Machines (CBMM), 2023-02-14)
      In this paper, we investigate the Rademacher complexity of deep sparse neural networks, where each neuron receives a small number of inputs. We prove generalization bounds for multilayered sparse ReLU neural networks, ...