“Yes, but will it work for my patients?” Driving clinically relevant research with benchmark datasets
Name
s41746-020-0295-6.pdf
Description
Published version
Size
438.75 KB
Format
Adobe PDF
Checksum (MD5)
d27938cd7dff0495a3340cca0a62c707
Author(s)
Panch, Trishan
Pollard, Tom Joseph
Mattie, Heather
Lindemer, Emily
Keane, Pearse A.
Celi, Leo Anthony G.
Date Issued
June 2020
Journal
npj Digital Medicine
Publisher
Springer Science and Business Media LLC
Citation
Panch, Trishan et al. "“Yes, but will it work for my patients?” Driving clinically relevant research with benchmark datasets." npj Digital Medicine 3, 1 (June 2020): 87 © 2020 The Author(s)
Version
Final published version
Abstract
Benchmark datasets have a powerful normative influence: by determining how the real world is represented in data, they define which problems will first be solved by algorithms built using the datasets and, by extension, who these algorithms will work for. It is desirable for these datasets to serve four functions: (1) enabling the creation of clinically relevant algorithms; (2) facilitating like-for-like comparison of algorithmic performance; (3) ensuring reproducibility of algorithms; (4) asserting a normative influence on the clinical domains and diversity of patients that will potentially benefit from technological advances. Without benchmark datasets that satisfy these functions, it is impossible to address two perennial concerns of clinicians experienced in computational research: “the data scientists just go where the data is rather than where the needs are,” and, “yes, but will this work for my patients?” If algorithms are to be developed and applied for the care of patients, then it is prudent for the research community to create benchmark datasets proactively, across specialties. As yet, best practice in this area has not been defined. Broadly speaking, efforts will include design of the dataset; compliance and contracting issues relating to the sharing of sensitive data; enabling access and reuse; and planning for translation of algorithms to the clinical environment. If a deliberate and systematic approach is not followed, not only will the considerable benefits of clinical algorithms fail to be realized, but the potential harms may be regressively incurred across existing gradients of social inequity.
MIT Department
Massachusetts Institute of Technology. Institute for Medical Engineering & Science
Terms of Use
Creative Commons Attribution 4.0 International license
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1038/s41746-020-0295-6
