Browsing Publications by Author "Siegel, Zachary"

Now showing items 1-1 of 1

    • SGD and Weight Decay Provably Induce a Low-Rank Bias in Deep Neural Networks 

      Galanti, Tomer; Siegel, Zachary; Gupte, Aparna; Poggio, Tomaso (Center for Brains, Minds and Machines (CBMM), 2023-02-14)
      In this paper, we study the bias of Stochastic Gradient Descent (SGD) to learn low-rank weight matrices when training deep ReLU neural networks. Our results show that training neural networks with mini-batch SGD and weight ...