Learning from partially labeled data

Abstract

Classification with partially labeled data involves learning from a few labeled examples as well as a large number of unlabeled examples, and represents a blend of supervised and unsupervised learning. Unlabeled examples provide information about the input domain distribution, but only the labeled examples indicate the actual classification task. The key question is how to improve classification accuracy by linking aspects of the input distribution P(x) to the conditional output distribution P(yx ) of the classifier. This thesis presents three approaches to the problem, starting with a kernel classifier that can be interpreted as a discriminative kernel density estimator and is trained via the EM algorithm or via margin-based criteria. Secondly, we employ a Markov random walk representation that exploits clusters and low-dimensional structure in the data in a robust and probabilistic manner. Thirdly, we introduce information regularization, a non-parametric technique based on minimizing information about labels over regions covering the domain. Information regularization provides a direct and principled way of linking P(x) to P(yx), and remains tractable for continuous P(x). The partially labeled problem arises in many applications where it is easy to collect unlabeled examples, but labor-intensive to classify the examples. The thesis demonstrates that the approaches require very few labeled examples for high classification accuracy on text and image-classification tasks.