The Geometry of Concepts: Sparse Autoencoder Feature Structure
Name
entropy-27-00344.pdf
Size
2.58 MB
Format
Adobe PDF
Checksum (MD5)
3bc38ce2cde4e9657af6e742985db4a4
Author(s)
Li, Yuxiao
Michaud, Eric J.
Baek, David D.
Engels, Joshua
Sun, Xiaoqing
Tegmark, Max
Date Issued
March 27, 2025
Journal
Entropy
Publisher
Multidisciplinary Digital Publishing Institute
Citation
Li, Y.; Michaud, E.J.; Baek, D.D.; Engels, J.; Sun, X.; Tegmark, M. The Geometry of Concepts: Sparse Autoencoder Feature Structure. Entropy 2025, 27, 344.
Version
Final published version
Abstract
Sparse autoencoders have recently produced dictionaries of high-dimensional vectors corresponding to the universe of concepts represented by large language models. We find that this concept universe has interesting structure at three levels: (1) The “atomic” small-scale structure contains “crystals” whose faces are parallelograms or trapezoids, generalizing well-known examples such as (man:woman::king:queen). We find that the quality of such parallelograms and associated function vectors improves greatly when projecting out global distractor directions such as word length, which is efficiently performed with linear discriminant analysis. (2) The “brain” intermediate-scale structure has significant spatial modularity; for example, math and code features form a “lobe” akin to functional lobes seen in neural fMRI images. We quantify the spatial locality of these lobes with multiple metrics and find that clusters of co-occurring features, at coarse enough scale, also cluster together spatially far more than one would expect if feature geometry were random. (3) The “galaxy”-scale large-scale structure of the feature point cloud is not isotropic, but instead has a power law of eigenvalues with steepest slope in middle layers. We also quantify how the clustering entropy depends on the layer.
MIT Department
Massachusetts Institute of Technology. Department of Physics
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.3390/e27040344
