Perspective Plane Program Induction From a Single Image
Name
2006.14708.pdf
Description
Accepted version
Size
7.57 MB
Format
Unknown
Checksum (MD5)
20f4e7453122c9f8b79fe38f21e5588c
Author(s)
Li, Yikai
Mao, Jiayuan
Zhang, Xiuming
Freeman, William T.
Tenenbaum, Joshua B.
Wu, Jiajun
Date Issued
August 2020
Journal
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Li, Yikai, Mao, Jiayuan, Zhang, Xiuming, Freeman, William T, Tenenbaum, Joshua B et al. 2020. "Perspective Plane Program Induction From a Single Image." Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.
Version
Author's final manuscript
Abstract
© 2020 IEEE. We study the inverse graphics problem of inferring a holistic representation for natural images. Given an input image, our goal is to induce a neuro-symbolic, program-like representation that jointly models camera poses, object locations, and global scene structures. Such high-level, holistic scene representations further facilitate low-level image manipulation tasks such as inpainting. We formulate this problem as jointly finding the camera pose and scene structure that best describe the input image. The benefits of such joint inference are two-fold: scene regularity serves as a new cue for perspective correction, and in turn, correct perspective correction leads to a simplified scene structure, similar to how the correct shape leads to the most regular texture in shape from texture. Our proposed framework, Perspective Plane Program Induction (P3I), combines search-based and gradient-based algorithms to efficiently solve the problem. P3I outperforms a set of baselines on a collection of Internet images, across tasks including camera pose estimation, global structure inference, and down-stream image manipulation tasks.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/cvpr42600.2020.00449
