Adaptive Machine Learning
Name
peroni-mperoni1-phd-orc-2026-thesis.pdf
Size
15.12 MB
Format
Adobe PDF
Checksum (MD5)
464bf722cc5a5035dffd4ad3335183ea
Author(s)
Peroni, Matthew Hamilton
Advisor(s)
Bertsimas, Dimitris J.
Date Issued
May 2026
Publisher
Massachusetts Institute of Technology
Abstract
Modern machine learning systems are increasingly deployed in settings where predictive performance must be balanced against real-world constraints, including limited data availability, heterogeneous acquisition costs, and the need for robustness in high-stakes decision-making. This thesis develops a unified framework for adaptive machine learning in which models dynamically allocate computational, modeling, and data resources based on the context of each input or task. The central premise is that, rather than applying uniform computation or relying on fixed pipelines, machine learning systems can achieve improved performance, robustness, and operational efficiency by learning when, where, and how to compute. Further, we aim to demonstrate that these principles are relevant for both traditional ML methods, such as decision trees, as well as cutting-edge foundation models. This thesis explores adaptivity across multiple levels of the machine learning pipeline. The chapter ordering reflects a natural progression from external decision-making around existing predictive models to increasingly internal forms of adaptivity. First, we develop Optimal Predictive-Policy Trees (OP²T), an interpretable framework for adaptive model selection that routes between models, forms ensembles, or abstains from prediction using transparent decision policies. Second, we propose Efficient Adaptive Modality Acquisition (EAMA), a framework for per-sample, multi-stage selection of costly modalities based on a recursive, energy-based value function that estimates the utility of acquiring subsets of unobserved modalities conditioned on those already observed, incorporates a principled selection rule based on model confusion rate, and enables efficient optimization by exploiting submodular structure. Third, we introduce Robust Tabular Foundation Models (RTFM), a framework for robust tabular foundation model pretraining via adversarial, difficulty-aware synthetic data generation that emphasizes challenging regions of the data space to improve generalization. Finally, we present adaptive architectures for efficient deep learning, including our work on DeepTrees and Skip Transformers, which selectively activate components of a model based on the input, applied to soft decision trees and transformers respectively. Together, these contributions establish a unified perspective on adaptive machine learning spanning model selection, data acquisition, training, and architecture design. By explicitly incorporating resource constraints and uncertainty into the learning process, this work advances the development of machine learning systems that are not only accurate, but also efficient, interpretable, and robust in deployment. This perspective highlights key overlap between modern ML/AI systems and operations research, where decision-making systems must optimize the use of scarce resources under uncertainty, demonstrating how concepts and techniques from OR can enable adaptive machine learning systems that do more with less in real-world applications.
MIT Department
Massachusetts Institute of Technology. Operations Research Center
Sloan School of Management
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
