Preventing Opioid Overdose: From Prediction to Operationalization

Abstract

The opioid epidemic remains a significant public health challenge in the US. A potential catalyst for reducing the incidence of opioid-related harm is the development and operationalization of risk stratification models. Prior work has focused on the statistical performance of such models without considering operational implications. Predicting the most severe outcome (fatal overdose) is a particular challenge due to imbalanced datasets. We partner with Staten Island Performing Provider System to access claims data and electronic health records for the patient population on Staten Island. For this population, we develop a single machine learning model for predicting a full range of adverse opioid-related events, and achieve an area under the receiver operating characteristic curve of 0.95, 0.87, 0.83 for the outcomes of any adverse opioid event, opioid overdose, and fatal opioid overdose, respectively, even in the absence of training data on fatal overdoses. Subsequently, we conduct a rolling horizon analysis to evaluate the capacity requirements of intervention policies leveraging the model. We find that the model can be used to identify a small intervention cohort (1% of the highest-risk patients) which includes the majority (69%) of adverse opioid events, allowing for targeted interventions with limited intervention capacity. Finally, we quantify the tradeoff between predictive performance and concerns that arise in implementation, such as interpretability, delay in data feeds, and prediction window length. Our results suggest that predictive performance does not need to be sacrificed to satisfy implementation concerns.