Mechanistic Interpretability of Genome-Scale DNA Large Language Models
Name
latawa-platawa-meng-eecs-2026-thesis.pdf
Size
104.52 MB
Format
Adobe PDF
Checksum (MD5)
a836a814fcecd5db344ed8bb7b3d4659
Author(s)
Latawa, Paridhi
Advisor(s)
Zhang, Feng
Date Issued
May 2026
Publisher
Massachusetts Institute of Technology
Abstract
Genome mining has enabled the discovery of diverse molecular systems within previously uncharacterized regions of DNA. The non-coding genome is of particular interest because it contains regulatory elements, structural RNAs, and other hidden functional signals that may give rise to previously unknown biological systems. Recent DNA foundation models show that some of this complexity can be encoded directly within the sequence. Models such as Evo 2 predict regulatory activity and variant effects with high accuracy, indicating that their internal high-dimensional representations learn functional patterns. While these models have been deployed as predictors, their internal signals remain largely unexplored as interpretable frameworks for biological discovery, especially in non-coding regions. In this thesis, we use two signals from Evo 2 for genome discovery: Shannon entropy, which highlights unexpected nucleotide positions, and sparse autoencoder features over internal representations, which characterize what biological structures these positions encode.
We develop a pipeline that scores entropy from Evo 2 across whole genomes and detects boundaries where the model’s confidence shifts sharply, which denotes transitions where the genome’s composition changes. We then extract sparse autoencoder activations from these regions, embed them into a shared latent space, and cluster them to reveal groups of regions with common biological functions, recovering canonical annotation classes without supervision. Through annotation-by-inference, we can identify unannotated functional regions that share latent space with known biological classes, demonstrating that DNA language models can serve as discovery engines for the unexplored genome.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link