Predictive and prescriptive methods in operations research and machine learning : an optimization approach

Abstract

The availability and prevalence of data have provided a substantial opportunity for decision makers to improve decisions and outcomes by effectively using this data. In this thesis, we propose approaches that start from data leading to high-quality decisions and predictions in various application areas. In the first chapter, we consider problems with observational data, and propose variants of machine learning (ML) algorithms that are trained by taking into account decision quality. The traditional approach to such a task has often focused on two-steps, separating the estimation task from the subsequent optimization task which uses these estimated models. Consequently, this approach can miss out on potential improvements in decision quality by considering these tasks jointly. Crucially, this leads to stronger prescriptive performance, particularly for smaller training set sizes, and improves the decision quality by 3 - 5% over other state-of-the-art methods.

We introduce the idea of uncertainty penalization to control the optimism of these methods which improves their performance, and propose finite-sample regret bounds. Through experiments on real and synthetic data sets, we demonstrate the value of this approach. In the second chapter, we consider observational data with decision-dependent uncertainty; in particular, we focus on problems with a finite number of possible decisions (treatments). We present our method of prescriptive trees, that prescribes the best treatment option by learning from observational data while simultaneously predicting counterfactuals. We demonstrate the effectiveness of such an approach using real data for the problem of personalized diabetes management. In the third chapter, we consider stochastic optimization problems when the sample average approximation approach is computationally expensive.

We introduce a novel measure, called the Prescriptive divergence which takes into account the decision quality of the scenarios, and consider scenario reduction in this context. We demonstrate the power of this optimization-based approach on various examples. In the fourth chapter, we present our work on a problem in predictive analytics where we focus on ML problems from a modern optimization perspective. For sparse shape-constrained regression problems, we propose modern optimization based algorithms that are scalable, and recover the true support with high accuracy and low false positive rates.