All photons imaging : time-resolved computational imaging through scattering for vehicles and medical applications with probabilistic and data-driven algorithms

Abstract

One of the greatest challenges in computational imaging is scaling it to work outside the lab. The main reasons for that challenge are the strong dependency on precise calibration, accurate physical models, and long acquisition times. These prevent practical progress towards medical imaging and seeing through occlusions such as fog in the wild. This dissertation demonstrates that with data-driven and probabilistic modeling we can alleviate these dependencies, and pave the way towards real-world time-resolved computational imaging through extreme scattering conditions using visible light. The ability to image through scattering media in the visible part of the electromagnetic spectrum holds many applications in various industries. For example, seeing through fog would enable autonomous robots to operate in challenging weather conditions; augment human driving; and allow airplanes, helicopters, and drones to take off and land in dense fog conditions.

In medical imaging, the ability to see into the body with near-infrared light would reduce the exposure to ionizing radiation and provide more clinically meaningful data. In order to image in diverse and extreme scattering conditions, we develop novel algorithms inspired by techniques in signal processing, optimization, statistical analysis, compressive sensing, and machine learning that leverage time-resolved sensing. More specifically, we demonstrate techniques that computationally leverage all of the optical signal, including scattered light, as opposed to locking onto a specific part of the optical signal. Furthermore, we show that by introducing probabilistic formulation to the imaging problem, the resulting system does not require user input for calibration and priors; this makes our systems more practical for real-world scenarios and enables them to operate in a wide range of scattering conditions.

We consider four cases of imaging through scattering media with increasing complexity: 1. A theoretical analysis of time-resolved single pixel imaging, which demonstrates scene reconstruction even when the entire scene is measured with a single pixel, an equivalent of simple scattering or a blur that is easy to model. 2. A data-driven calibration invariant technique for imaging through simple scattering (a sheet of paper). 3. Imaging through a thick tissue phantom by utilizing all of the optical signal with minimal assumptions on the tissue properties. 4. Imaging through a wide range of dense, dynamic, and heterogeneous fog conditions. In that case, we introduce a probabilistic model that is able to recover the occluded target reflectance and depth without any assumption about the fog.