Generalization of deep neural networks to unseen attribute combinations

Abstract

Visual understanding results from a combined understanding of primitive visual attributes such as color, texture, and shape. This allows humans and other primates to generalize their understanding of objects to new combinations of attributes. For instance, one can understand that a pink elephant is an elephant even if they have never seen this particular combination of color and shape before. However, is it the case that deep neural networks (DNNs) are able to generalize to such novel combinations in object recognition or other related vision tasks? This thesis demonstrates that (1) the ability of DNNs to generalize to unseen attribute combinations increases with the increased diversity of combinations seen in training as a percentage of the total combination space, (2) this effect is largely independent of the specifics of the DNN architecture used, (3) while single-task and multi-task formulations of supervised attribute classification problems may lead to similar performance on seen combinations, single-task formulations have a superior ability to generalize to unseen combinations, and (4) DNNs demonstrating the ability to generalize well in this setting learn to do so by leveraging emergent hidden units that exhibit properties of attribute selectivity and invariance.