Data-Efficient Machine Learning with Applications to Cardiology

Abstract

Deep learning models have demonstrated impressive capabilities in many settings including computer vision, natural language generation, and speech processing. However, an important shortcoming of these models is that they often need to be trained on large datasets in order to be most effective. In domains such as medicine, large datasets are not always available, and thus there is a need for data-efficient models that perform well even in limited data regimes. In this thesis, motivated by this need, we present four contributions to data-efficient machine learning: (1) analyzing and improving few-shot learning, where we study a popular few-shot learning algorithm (Model Agnostic Meta-Learning) and provide insights as to why it is effective, proposing a simplified version that offers substantial computational benefits; (2) improving supervised learning on small clinical datasets of electrocardiograms (ECGs), where we develop a new data augmentation strategy for ECGs that helps boost performance on a range of predictive problems; (3) improving pre-training through the use of nested optimization, introducing an efficient gradient based algorithm to jointly optimize model parameters and pre-training algorithm design choices; and (4) developing a new self-supervised learning pipeline for complex clinical time series, where the design of the pipeline is driven by the multimodal, multi-dimensional nature of real-world clinical time series data. Unifying several of these contributions is the application area of cardiovascular medicine, a setting in which machine learning has the potential to improve patient care and outcomes.