What Disease Does This Patient Have? A Large-Scale Open Domain Question Answering Dataset from Medical Exams
Author(s)
Jin, Di; Pan, Eileen; Oufattole, Nassim; Weng, Wei-Hung; Fang, Hanyi; Szolovits, Peter; ... Show more Show less
Downloadapplsci-11-06421-v4.pdf (302.6Kb)
Publisher with Creative Commons License
Publisher with Creative Commons License
Creative Commons Attribution
Terms of use
Metadata
Show full item recordAbstract
Open domain question answering (OpenQA) tasks have been recently attracting more and more attention from the natural language processing (NLP) community. In this work, we present the first free-form multiple-choice OpenQA dataset for solving medical problems, <span style="font-variant: small-caps;">MedQA</span>, collected from the professional medical board exams. It covers three languages: English, simplified Chinese, and traditional Chinese, and contains 12,723, 34,251, and 14,123 questions for the three languages, respectively. We implement both rule-based and popular neural methods by sequentially combining a document retriever and a machine comprehension model. Through experiments, we find that even the current best method can only achieve 36.7%, 42.0%, and 70.1% of test accuracy on the English, traditional Chinese, and simplified Chinese questions, respectively. We expect <span style="font-variant: small-caps;">MedQA</span> to present great challenges to existing OpenQA systems and hope that it can serve as a platform to promote much stronger OpenQA models from the NLP community in the future.
Date issued
2021-07Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence LaboratoryJournal
Applied Sciences
Publisher
Multidisciplinary Digital Publishing Institute
Citation
Applied Sciences 11 (14): 6421 (2021)
Version: Final published version
ISSN
2076-3417