Real-world deep domain adaptation through prediction propagation

Abstract

Deep learning-based natural language processing classifiers often have difficulty classifying texts that stem from a different domain than the labeled training data. Many domain adaptation methods have been proposed to train classifiers using only labeled texts from a single domain and unlabeled texts from other domains. Nevertheless, we find that the state-of-the-art methods all lack one or more desirable properties for real-world modeling. In particular, we find that the many methods using a domain-adversarial loss are unable to model domains with different label distributions. Motivated by these limitations, we are developing a new method, Prediction Propagation, that can classify texts from different domains without using an adversarial loss. Our method will use the label prediction for reconstructing the input text and backpropagates through the prediction as a way to learn label-related information for the new domain. Our method has the desirable properties for real-world modeling while not compromising on performance.