Shadows, Mirrors, and Snow: Decoding Multibounce Light Transport in Time-of-Flight Photography

Abstract

Time-of-flight photography is a new imaging modality that combines the precise time-of-flight measurements of lidar with the wide field of view and high angular resolution of photography. The result of this combination is that time-of-flight cameras by nature capture videos of light as it flows through the photographed scene. Unlike regular photographs, time-of-flight photographs expose the sequence of how light interacts with a scene. Furthermore, the fixed and finite speed of light in free space tightly couples the dimensions of space and time, such that it is typically straightforward to determine precisely where and when observed scattering events occur.

In this thesis we present five new imaging methods that leverage the unique advantages of time-of-flight photography to analyze light transport within a scene and, through this analysis, infer scene properties. First, we show that shadows observed in light that has scattered exactly twice before returning to the camera can be used to infer the geometry of surfaces that are hidden from view. Second, we show how the time-of-flight of these two-bounce returns can be used to retrieve the visible scene’s geometry. Third, we propose an algorithm to retrieve the spatially varying BRDF of visible surfaces from two-bounce returns. Fourth, we show how multi-bounce returns measured with a time-of-flight camera can be used to unambiguously detect and localize specular surfaces like mirrors and windows. Finally, we show how time-resolved measurements of light that scatters many times within snow before returning to the camera can be used to infer the snow’s density and grain size, and the concentrations of impurities within the snow.

Our hope is that introducing several useful and practical applications of time-of-flight photography will spur interest and motivate further research into new applications and the fundamental principles of this exciting new technology.