Modular Expert Injection for Scalable Class-Incremental Classification Using Frozen Foundation Model Features
Name
Final_Paper_McCormick.pdf
Size
714.77 KB
Format
Adobe PDF
Checksum (MD5)
9d588ea354154b172a24f3b21ef63ce9
Author(s)
Mccormick, Gavin
Smith, Liam
Botero, Joey
Date Issued
August 20, 2026
Abstract
A modular classification framework that supports the incremental addition of new classes without retraining previously deployed components has been developed. The system maintains one small expert network per class, each trained as a binary one-versus-rest classifier over features extracted from a frozen DINOv2 vision transformer backbone. A transformerbased fusion module combines the outputs of all experts to produce the final multi-class prediction. When a new class needs to be added, a new expert is trained and the fusion head accepts its features without modifying any prior expert weights. Training the new expert uses both the incoming class data as well as a replay of the original training data, providing the fusion head sufficient context to assign boundaries across all existing classes. Experiments on a five-class ImageNet subset demonstrate a baseline accuracy of 85.77% prior to the new class being injected. After injecting the sixth class, the system achieves 81.88% overall accuracy across all classes. The sixth class demonstrated an accuracy of 74.91% and previous classes suffering only modest degradation. The proposed approach addresses a practical gap in operational classification pipelines where new categories must be rapidly deployed, and full retraining is computationally or operationally impractical.
Subjects
Class-incremental learning
mixture of experts
continual learning
catastrophic forgetting
foundation models
DINOv2
modular classification
transfer learning
MIT Department
Lincoln Laboratory
Terms of Use
All rights reserved
Persistent DSpace Link
