The perception of material qualities in real-world images

Abstract

One can easily tell if a sidewalk is slippery, if food is fresh, if a spoon is made of plastic or stainless steel, or if a suspicious looking mole warrants a trip to the doctor. This ability to visually identify and discriminate materials is known as material perception and little is known about it. We have measured human material judgments on a wide range of complex, real world materials. We have gathered several diverse image databases and made use of them to conduct psychophysical studies. We asked observers to classify surfaces and objects as being made of fabric, paper, plastic or other common material categories. In the first part of this thesis, we present experiments that establish that observers can make these judgments of material category reliably, quickly and in challenging conditions of rapid presentation. We find that categorization performance cannot be explained by simple, low-level cues like color or high spatial frequencies. In the second part of the thesis, we explore judgments beyond those of common material categories. Observers judged many dimensions of material appearance such as matte vs. glossy, opaque vs. translucent, rigid vs. nonrigid, soft vs. rough to touch, and even genuine vs. fake for familiar object categories like flowers, fruits and dessert. Observers were surprisingly accurate, even in 40 millisecond presentations. In the final part of this thesis, we compare the performance of state-of-art computer vision techniques with human performance on our images and tasks and find current techniques to be severely lacking.

(cont.) Taken together, our findings indicate that material perception is a distinct mechanism and can be as fast and flexible as object recognition or scene perception. When recognizing materials, low-level image information is of limited use for both humans and computer vision systems. We conclude that material recognition is a rich and challenging problem domain and there is much ground to be covered in both visual perception and computer vision.