Visual Deprojection: Probabilistic Recovery of Collapsed Dimensions
Name
1909.00475.pdf
Description
Submitted version
Size
2.07 MB
Format
Adobe PDF
Checksum (MD5)
7cee2fe92fa1d108deb72999b45a7e51
Author(s)
Balakrishnan, Guha
Dalca, Adrian Vasile
Zhao, Amy (Xiaoyu Amy)
Guttag, John V
Durand, Frédo
Freeman, William T
Date Issued
October 2019
Journal
Proceedings of the IEEE International Conference on Computer Vision
Publisher
IEEE
Citation
Balakrishnan, Guha et al. “Visual Deprojection: Probabilistic Recovery of Collapsed Dimensions.” Paper in the Proceedings of the IEEE International Conference on Computer Vision, 2019-October, 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), 27 October-2 November 2019, IEEE © 2019 The Author(s)
Version
Original manuscript
Abstract
We introduce visual deprojection: The task of recovering an image or video that has been collapsed along a dimension. Projections arise in various contexts, such as long-exposure photography, where a dynamic scene is collapsed in time to produce a motion-blurred image, and corner cameras, where reflected light from a scene is collapsed along a spatial dimension because of an edge occluder to yield a 1D video. Deprojection is ill-posed - often there are many plausible solutions for a given input. We first propose a probabilistic model capturing the ambiguity of the task. We then present a variational inference strategy using convolutional neural networks as functional approximators. Sampling from the inference network at test time yields plausible candidates from the distribution of original signals that are consistent with a given input projection. We evaluate the method on several datasets for both spatial and temporal deprojection tasks. We first demonstrate the method can recover human gait videos and face images from spatial projections, and then show that it can recover videos of moving digits from dramatically motion-blurred images obtained via temporal projection.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/ICCV.2019.00026
