Self-Training for Natural Language Processing
Name
Luo-hyluo-PhD-EECS-2022-thesis.pdf
Description
Thesis PDF
Size
3.22 MB
Format
Adobe PDF
Checksum (MD5)
d59b5e4c0aaa6d2668601fedd6fccf66
Author(s)
Luo, Hongyin
Advisor(s)
Glass, James R.
Date Issued
May 2022
Publisher
Massachusetts Institute of Technology
Abstract
Data annotation is critical for machine learning based natural language processing models. Although many large-scale corpora and standard benchmarks have been annotated and published, they cannot cover all possible applications. As a result, it is difficult to transfer models trained with public corpora to tasks that require domain-specific knowledge, different inference skills, unseen text styles, and explainability. In this thesis, we explore self-training methods for mitigating the data distribution gaps between training and evaluation domains and tasks. In contrast to traditional self-training methods that study the best practice of training models with real data and pseudo labels, we also explore the possibility of automatically generating synthetic data for better explainability, robustness, and domain adaptation performance. We show the performance improvement achieved by our methods on different natural language understanding and generation tasks, including question answering, question generation, and dialog response selection.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright MIT
Persistent DSpace Link
