Human visual perception under real-world illumination

Abstract

How does the visual system achieve stable estimates of surface properties - such as reflectance and 3D shape - across changes in the illumination? Under arbitrary patterns of illumination this problem is ill-posed. However, in the real world, illumination is not arbitrary. Here I argue that the visual system exploits the statistical regularities of real-world illuminations to achieve stable estimates of shape and surface reflectance properties. Specifically, I suggest that the visual system derives measurements from specular reflections that are (i) diagnostic of surface properties and (ii) relatively well-conserved across real-world scenes. One consequence of the theory is that the visual system does not have to estimate and explicitly 'discount' illumination to recover shape and surface reflectance. In support of this idea, subjects are shown to be good at estimating surface reflectance and 3D shape without any context to specify the surrounding scene, as long as the illumination is realistic. However, when the pattern of illumination is unrealistic, shape and surface reflectance estimation degrade in predictable ways. Systematic manipulation of illumination statistics reveals some properties of illumination that are important for surface reflectance estimation. To understand 3D shape constancy, I discuss the way that 3D surface curvature distorts the reflected world. For the special case of mirrored surfaces, I show how populations of oriented linear filters can 'read' the pattern of distortions to recover 3D surface curvatures. Finally I show that this principle applies to cases other than perfect mirrors, and can predict both successes and failures of human shape constancy as the illumination changes.