Seeing Is Believing: Brain-Inspired Modular Training for Mechanistic Interpretability
Name
entropy-26-00041-v2.pdf
Size
4.55 MB
Format
Adobe PDF
Checksum (MD5)
fc897a75c30e5873aba8a0da6a5df570
Author(s)
Liu, Ziming
Gan, Eric
Tegmark, Max
Date Issued
December 30, 2023
Publisher
Multidisciplinary Digital Publishing Institute
Citation
Entropy 26 (1): 41 (2024)
Version
Final published version
Abstract
We introduce Brain-Inspired Modular Training (BIMT), a method for making neural networks more modular and interpretable. Inspired by brains, BIMT embeds neurons in a geometric space and augments the loss function with a cost proportional to the length of each neuron connection. This is inspired by the idea of minimum connection cost in evolutionary biology, but we are the first the combine this idea with training neural networks with gradient descent for interpretability. We demonstrate that BIMT discovers useful modular neural networks for many simple tasks, revealing compositional structures in symbolic formulas, interpretable decision boundaries and features for classification, and mathematical structure in algorithmic datasets. Qualitatively, BIMT-trained networks have modules readily identifiable by the naked eye, but regularly trained networks seem much more complicated. Quantitatively, we use Newman’s method to compute the modularity of network graphs; BIMT achieves the highest modularity for all our test problems. A promising and ambitious future direction is to apply the proposed method to understand large models for vision, language, and science.
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.3390/e26010041
