Hard for humans, hard for machines: predicting readmission after psychiatric hospitalization using narrative notes
Name
s41398-020-01104-w.pdf
Description
Published version
Size
505.18 KB
Format
Adobe PDF
Checksum (MD5)
9b8b792b2105d138ec0ab3b1bce82eb3
Author(s)
Boag, William
Kovaleva, Olga
McCoy, Thomas H
Rumshisky, Anna
Szolovits, Peter
Perlis, Roy H
Date Issued
2021
Journal
Translational Psychiatry
Publisher
Springer Science and Business Media LLC
Version
Final published version
Abstract
© 2021, The Author(s). Machine learning has been suggested as a means of identifying individuals at greatest risk for hospital readmission, including psychiatric readmission. We sought to compare the performance of predictive models that use interpretable representations derived via topic modeling to the performance of human experts and nonexperts. We examined all 5076 admissions to a general psychiatry inpatient unit between 2009 and 2016 using electronic health records. We developed multiple models to predict 180-day readmission for these admissions based on features derived from narrative discharge summaries, augmented by baseline sociodemographic and clinical features. We developed models using a training set comprising 70% of the cohort and evaluated on the remaining 30%. Baseline models using demographic features for prediction achieved an area under the curve (AUC) of 0.675 [95% CI 0.674–0.676] on an independent testing set, while language-based models also incorporating bag-of-words features, discharge summaries topics identified by Latent Dirichlet allocation (LDA), and prior psychiatric admissions achieved AUC of 0.726 [95% CI 0.725–0.727]. To characterize the difficulty of the task, we also compared the performance of these classifiers to both expert and nonexpert human raters, with and without feedback, on a subset of 75 test cases. These models outperformed humans on average, including predictions by experienced psychiatrists. Typical note tokens or topics associated with readmission risk were related to pregnancy/postpartum state, family relationships, and psychosis.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Creative Commons Attribution 4.0 International license
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1038/s41398-020-01104-w
