This is not the latest version of this item. The latest version can be found here.
End-to-End Optimization of Scene Layout
Name
3dsln_cvpr.pdf
Description
Accepted version
Size
1.06 MB
Format
Adobe PDF
Checksum (MD5)
2c2fa8ef8a9a716a466bae1f77162cbf
Author(s)
Luo, Andrew
Zhang, Zhoutong
Wu, Jiajun
Tenenbaum, Joshua B
Date Issued
2020
Journal
Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Luo, Andrew, Zhang, Zhoutong, Wu, Jiajun and Tenenbaum, Joshua B. 2020. "End-to-End Optimization of Scene Layout." Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.
Version
Author's final manuscript
Abstract
© 2020 IEEE. We propose an end-to-end variational generative model for scene layout synthesis conditioned on scene graphs. Unlike unconditional scene layout generation, we use scene graphs as an abstract but general representation to guide the synthesis of diverse scene layouts that satisfy relationships included in the scene graph. This gives rise to more flexible control over the synthesis process, allowing various forms of inputs such as scene layouts extracted from sentences or inferred from a single color image. Using our conditional layout synthesizer, we can generate various layouts that share the same structure of the input example. In addition to this conditional generation design, we also integrate a differentiable rendering module that enables layout refinement using only 2D projections of the scene. Given a depth and a semantics map, the differentiable rendering module enables optimizing over the synthesized layout to fit the given input in an analysis-by-synthesis fashion. Experiments suggest that our model achieves higher accuracy and diversity in conditional scene synthesis and allows exemplar-based scene generation from various input forms.
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
10.1109/CVPR42600.2020.00381
