Formation of Representations in Neural Networks
Name
CBMM-Memo-150.pdf
Size
4.03 MB
Format
Adobe PDF
Checksum (MD5)
6481acadcf5b7c2cf3cbc42a8f2d503f
Author(s)
Ziyin, Liu
Chuang, Isaac
Galanti, Tomer
Poggio, Tomaso
Date Issued
October 7, 2024
Publisher
Center for Brains, Minds and Machines (CBMM)
Series/Report no.
CBMM Memo;150
Abstract
Understanding neural representations will help open the black box of neural networks and advance our scientific understanding of modern AI systems. However, how complex, structured, and transferable representations emerge in modern neural networks has remained a mystery. Building on previous results, we propose the Canonical Representation Hypothesis (CRH), which posits a set of six alignment relations to universally govern the formation of representations in most hidden layers of a neural network. Under the CRH, the latent representations (R), weights (W), and neuron gradients (G) become mutually aligned during training. This alignment implies that neural networks naturally learn compact representations, where neurons and weights are invariant to task-irrelevant transformations. We then show that the breaking of CRH leads to the emergence of reciprocal power-law relations between R, W, and G, which we refer to as the Polynomial Alignment Hypothesis (PAH). We present a minimal-assumption theory demonstrating that the balance between gradient noise and regularization is crucial for the emergence the canonical representation. The CRH and PAH lead to an exciting possibility of unifying major key deep learning phenomena, including neural collapse and the neural feature ansatz, in a single framework.
Persistent DSpace Link
