Imputation of clinical covariates in time series
Name
10994_2020_5923_ReferencePDF.pdf
Size
1.13 MB
Format
Adobe PDF
Checksum (MD5)
362440e0dd0ea9093fa27e76780c0626
Author(s)
Bertsimas, Dimitris
Orfanoudaki, Agni
Pawlowski, Colin
Date Issued
November 10, 2020
Publisher
Springer US
Version
Author's final manuscript
Abstract
Abstract
Missing data is a common problem in longitudinal datasets which include multiple instances of the same individual observed at different points in time. We introduce a new approach, MedImpute, for imputing missing clinical covariates in multivariate panel data. This approach integrates patient specific information into an optimization formulation that can be adjusted for different imputation algorithms. We present the formulation for a K-nearest neighbors model and derive a corresponding scalable first-order method med.knn. Our algorithm provides imputations for datasets with both continuous and categorical features and observations occurring at arbitrary points in time. In computational experiments on three real-world clinical datasets, we test its performance on imputation and downstream predictive tasks, varying the percentage of missing data, the number of observations per patient, and the mechanism of missing data. The proposed method improves upon both the imputation accuracy and downstream predictive performance relative to the best of the benchmark imputation methods considered. We show that this edge is consistently present both in longitudinal and electronic health records datasets as well as in binary classification and regression settings. On computational experiments on synthetic data, we test the scalability of this algorithm on large datasets, and we show that an efficient method for hyperparameter tuning scales to datasets with 10,000’s of observations and 100’s of covariates while maintaining high imputation accuracy.
MIT Department
Massachusetts Institute of Technology. Operations Research Center
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1007/s10994-020-05923-2
