Pre-trained Language Models for Clinical Systematic Literature Reviews
Name
Ortiz-juanmoo-meng-eecs-2022-thesis.pdf
Description
Thesis PDF
Size
2.45 MB
Format
Adobe PDF
Checksum (MD5)
c743547d08d3adb232c8015eea7226a3
Author(s)
Ortiz, Juan M. Ochoa
Advisor(s)
Barzilay, Regina
Date Issued
February 2022
Publisher
Massachusetts Institute of Technology
Abstract
Although systematic literature reviews play a critical role in clinical-based decision making, manual methods for information extraction can sometimes take prohibitively long. In this work, we first describe the construction of datasets in two distinct clinical domains containing randomized trials and observational studies. We then utilize these two datasets to benchmark the performance of Pretrained Language Model (PLM) based entity and relation extraction models as well as the effect of domain specific pre-training prior to their fine-tuning. Our results show evidence to the effectiveness of pre-training using masked language modeling (MLM), a sentence-level proxy task, on boosting the performance of fine-tuned models on both inter- and intra-sentence level information extraction tasks.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright MIT
Persistent DSpace Link
