Representation Learning for Extrapolation via Bilinear Transduction

Abstract

Typical machine learning systems, such as deep neural networks, perform well at predicting on new examples that come from the same distribution as initial training data. However, these systems are not typically robust to examples that do not come from the same distribution as the training samples. These testing samples are characterized as out-of-distribution (OOD). Using a proven bilinear transduction [1] method for accurately predicting on OOD examples, we propose a method to apply this framework to learned representations instead of hand designed state representations. This work is geared towards enabling the bilinear transduction approach to generalize to a wider range of data types and tasks when such designed representations are not available. We use deep neural networks to learn representations of certain data types, such as images, and apply bilinear transduction to these learned representations. This has the potential to further expand the out-of-support prediction capabilities of the bilinear transduction framework.