Self-Training and Calibration for Learning with Limited Data
Name
Liu-emmaliu-meng-eecs-2022-thesis.pdf
Description
Thesis PDF
Size
1.95 MB
Format
Adobe PDF
Checksum (MD5)
9c51e19d8ec6f31e3813555b39373376
Author(s)
Liu, Emma J.
Advisor(s)
Wornell, Gregory W.
Sattigeri, Prasanna
Date Issued
May 2022
Publisher
Massachusetts Institute of Technology
Abstract
Semi-supervised learning methods such as self-training are able to leverage unlabeled data, which is widely available, as opposed to only using labeled data like many successful supervised learning methods. One part of self-training is to use a trained model to create pseudo-labels for unlabeled data and then select some of those samples to add to the labeled dataset. One way to do this is to pick samples for which the model has high confidence. However, many models are not well-calibrated, which means that the confidence scores do not necessarily align with the expected distribution in the dataset. Thus, the usage of confidence scores in this manner may result in adding more incorrectly labeled samples to the training dataset than expected. This thesis explores how the addition of a recalibration step during self-training to adjust the confidence scores before they are used to select samples can improve the results of self-training. Performing experiments on natural language processing data revealed that combining self-training with calibration results in improved accuracy when the initial self-training accuracy is not too high and the amount of labeled data initially used is not too small.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright MIT
Persistent DSpace Link
