Developing well-calibrated illness severity scores for decision support in the critically ill
Author(s)
Cosgriff, Christopher V.; Celi, Leo Anthony G.; Ko, Stephanie; Sundaresan, Tejas; Armengol de la Hoz, Miguel Ángel; Kaufman, Aaron Russell; Stone, David J; Badawi, Omar; Deliberato, Rodrigo; ... Show more Show less
Downloads41746-019-0153-6.pdf (1002.Kb)
Open Access Policy
Open Access Policy
Creative Commons Attribution-Noncommercial-Share Alike
Terms of use
Metadata
Show full item recordAbstract
Illness severity scores are regularly employed for quality improvement and benchmarking in the intensive care unit, but poor generalization performance, particularly with respect to probability calibration, has limited their use for decision support. These models tend to perform worse in patients at a high risk for mortality. We hypothesized that a sequential modeling approach wherein an initial regression model assigns risk and all patients deemed high risk then have their risk quantified by a second, high-risk-specific, regression model would result in a model with superior calibration across the risk spectrum. We compared this approach to a logistic regression model and a sophisticated machine learning approach, the gradient boosting machine. The sequential approach did not have an effect on the receiver operating characteristic curve or the precision-recall curve but resulted in improved reliability curves. The gradient boosting machine achieved a small improvement in discrimination performance and was similarly calibrated to the sequential models.
Date issued
2019-08Department
Harvard University--MIT Division of Health Sciences and Technology; Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory; Massachusetts Institute of Technology. Institute for Medical Engineering & Science; Harvard--MIT Program in Health Sciences and Technology. Laboratory for Computational Physiology; MIT Critical Data (Laboratory)Journal
npj Digital Medicine
Publisher
Springer Science and Business Media LLC
Citation
Cosgriff, C.V., Celi, L.A., Ko, S. et al. Developing well-calibrated illness severity scores for decision support in the critically ill. npj Digit. Med. 2, 76 (2019). © 2019 The Author(s)
Version: Final published version
ISSN
2398-6352