Interpretations of Machine Learning and Their Application to Therapeutic Design

Abstract

We introduce a framework for interpreting black-box machine learning (ML) models, discover overinterpretation as a failure mode of deep neural networks, and discuss how ML methods can be applied for therapeutic design, including a pan-variant COVID-19 vaccine. While ML models are widely deployed and often attain superior accuracy compared to traditional approaches, deep learning models are functionally complex and difficult to interpret, limiting their adoption in high-stakes environments. In addition to safer deployment, model interpretation also aids scientific discovery, where validated ML models trained on experimental data can be used to uncover biological mechanisms or to design therapeutics through biologically faithful objective functions, such as vaccine population coverage.

For interpretation of black-box ML models, we introduce the Sufficient Input Subsets (SIS) method that is model-agnostic, faithful to underlying functions, and conceptually straightforward. We demonstrate ML model interpretation with SIS in natural language, computer vision, and computational biological settings. Using the SIS framework, we discover overinterpretation, a novel failure mode of deep neural networks that can hinder generalizability in real-world environments. We posit that overinterpretation results from degenerate signals present in training datasets. Next, using ML models that have been calibrated with experimental immunogenicity data, we develop a flexible framework for the computational design of robust peptide vaccines. Our framework optimizes the n-times coverage of each individual in the population to activate broader T cell immune responses, account for differences in peptide immunogenicity across individuals, and reduce the chance of vaccine escape by mutations. Using this framework, we design vaccines for SARS-CoV-2 that have superior population coverage to published baselines and are conserved across variants of concern. We validate this approach in vivo through a COVID-19 animal challenge study of our vaccine. This thesis demonstrates distinct ways model interpretation enables ML methods to be faithfully deployed in biological settings.