Role of ordinal contrast relationships in face encoding

Abstract

What aspects of facial information do we use to recognize individuals? One way to address this fundamental question is to study image transformations that compromise facial recognizability. The goal would be to identify factors that underlie the recognition decrement and, by extension, are likely constituents of facial encoding. To this end, we focus here on the contrast negation transformation. Contrast negated faces are remarkably difficult to recognize for reasons that are currently unclear. The dominant proposals so far are based either on negative faces' seemingly unusual pigmentation, or incorrectly computed 3D shape. Both of these explanations have been challenged by recent results. Here, we propose an alternative account based on 2D ordinal relationships, which encode local contrast polarity between a few regions of the face. Using a novel set of facial stimuli that incorporate both positive and negative contrast, we demonstrate that ordinal relationships around the eyes are major determinants of facial recognizability. Our behavioral studies suggest that destruction of these relationships in negatives likely underlies the observed recognition impairments, and our neuro-imaging data show that these relationships strongly modulate brain responses to facial images. Besides offering a potential explanation for why negative faces are hard to recognize, these results have implications for the representational vocabulary the visual system uses to encode faces.