Deep Optical Coding Design in Computational Imaging: A data-driven framework
Name
2207.00164v2.pdf
Size
32.62 MB
Format
Adobe PDF
Checksum (MD5)
8a250876a8462b5f684ba72b08c0332b
Author(s)
Arguello, Henry
Bacca, Jorge
Kariyawasam, Hasindu
Vargas, Edwin
Marquez, Miguel
Hettiarachchi, Ramith
Garcia, Hans
Herath, Kithmini
Haputhanthri, Udith
Ahluwalia, Balpreet Singh
Date Issued
March 2023
Journal
IEEE Signal Processing Magazine
Publisher
Institute of Electrical and Electronics Engineers
Citation
Arguello, Henry, Bacca, Jorge, Kariyawasam, Hasindu, Vargas, Edwin, Marquez, Miguel et al. 2023. "Deep Optical Coding Design in Computational Imaging: A data-driven framework." IEEE Signal Processing Magazine, 40 (2).
Version
Author's final manuscript
Abstract
Computational optical imaging (COI) systems leverage optical coding elements (CE) in their setups to encode a high-dimensional scene in a single or multiple snapshots and decode it by using computational algorithms. The performance of COI systems highly depends on the design of its main components: the CE pattern and the computational method used to perform a given task. Conventional approaches rely on random patterns or analytical designs to set the distribution of the CE. However, the available data and algorithm capabilities of deep neural networks (DNNs) have opened a new horizon in CE data-driven designs that jointly consider the optical encoder and computational decoder. Specifically, by modeling the COI measurements through a fully differentiable image formation model that considers the physics-based propagation of light and its interaction with the CEs, the parameters that define the CE and the computational decoder can be optimized in an end-to-end (E2E) manner. Moreover, by optimizing just CEs in the same framework, inference tasks can be performed from pure optics. This work surveys the recent advances on CE data-driven design and provides guidelines on how to parametrize different optical elements to include them in the E2E framework. Since the E2E framework can handle different inference applications by changing the loss function and the DNN, we present low-level tasks such as spectral imaging reconstruction or high-level tasks such as pose estimation with privacy preserving enhanced by using optimal task-based optical architectures. Finally, we illustrate classification and 3D object recognition applications performed at the speed of the light using all-optics DNN.
MIT Department
Massachusetts Institute of Technology. Department of Mechanical Engineering
Massachusetts Institute of Technology. Department of Biological Engineering
Terms of Use
Creative Commons Attribution-Noncommercial-ShareAlike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/msp.2022.3200173
