New algorithms in machine learning with applications in personalized medicine

Abstract

Recent advances in machine learning and optimization hold much promise for influencing real-world decision making, especially in areas such as health care where abundant data are increasingly being collected. However, imperfections in the data pose a major challenge to realizing their full potential: missing values, noisy observations, and unobserved counterfactuals all impact the performance of data-driven methods. In this thesis, with a fresh perspective from optimization, I revisit some of the well-known problems in statistics and machine learning, and develop new methods for prescriptive analytics. I show examples of how common machine learning tasks, such as missing data imputation in Chapter 2 and classication in Chapter 3, can benet from the added edge of rigorous optimization formulations and solution techniques. In particular, the proposed opt.impute algorithm improves imputation quality by 13.7% over state-of-the-art methods, as averaged over 95 real data sets, which leads to further performance gains in downstream tasks. The power of prescriptive analytics is shown in Chapter 4 by our approach to personalized diabetes management, which identifies response patterns using machine learning and individualizes treatments via optimization. These newly developed machine learning algorithms not only demonstrate improved performance in large-scale experiments, but are also applied to solve the problems in health care that motivated them. Our simulated trial for diabetic patients in Chapter 4 demonstrates a clinically relevant reduction in average hemoglobin A1c levels compared to current practice. Finally, when predicting mortality for cancer patients in Chapter 5, applying opt.impute on missing data along with the cutting-edge algorithm Optimal Classication Tree on a rich data set prepared from electronic medical records, we are able to accurately risk stratify patients, providing physicians with interpretable insights and valuable risk estimates at time of treatment decisions and end-of-life planning.