MIT Libraries logoDSpace@MIT

MIT
View Item 
  • DSpace@MIT Home
  • MIT Open Access Articles
  • MIT Open Access Articles
  • View Item
  • DSpace@MIT Home
  • MIT Open Access Articles
  • MIT Open Access Articles
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

3D-aware scene manipulation via inverse graphics

Author(s)
Yao, Shunyu; Hsu, Tzu Ming; Zhu, Jun-Yan; Wu, Jiajun; Torralba, Antonio; Freeman, William T.; Tenenbaum, Joshua B.; ... Show more Show less
Thumbnail
DownloadPublished version (4.720Mb)
Terms of use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Metadata
Show full item record
Abstract
We aim to obtain an interpretable, expressive, and disentangled scene representation that contains comprehensive structural and textural information for each object. Previous scene representations learned by neural networks are often uninterpretable, limited to a single object, or lacking 3D knowledge. In this work, we propose 3D scene de-rendering networks (3D-SDN) to address the above issues by integrating disentangled representations for semantics, geometry, and appearance into a deep generative model. Our scene encoder performs inverse graphics, translating a scene into a structured object-wise representation. Our decoder has two components: a differentiable shape renderer and a neural texture generator. The disentanglement of semantics, geometry, and appearance supports 3D-aware scene manipulation, e.g., rotating and moving objects freely while keeping the consistent shape and texture, and changing the object appearance without affecting its shape. Experiments demonstrate that our editing scheme based on 3D-SDN is superior to its 2D counterpart. ©2018 Poster presentation at the 32nd annual Conference on Neural Information Processing Systems (NIPS 2018), December 3-5, 2018, Montréal, Québec.
Date issued
2018
URI
https://hdl.handle.net/1721.1/124516
Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science; Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Journal
Advances in Neural Information Processing Systems
Publisher
Neural Information Processing Systems Foundation, Inc.
Citation
Yao, Shunyu, et al., "3D-aware scene manipulation via inverse graphics." Advances in Neural Information Processing Systems 31 (2018) url https://papers.nips.cc/book/advances-in-neural-information-processing-systems-31-2018
Version: Final published version

Collections
  • MIT Open Access Articles

Browse

All of DSpaceCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

My Account

Login

Statistics

OA StatisticsStatistics by CountryStatistics by Department
MIT Libraries
PrivacyPermissionsAccessibilityContact us
MIT
Content created by the MIT Libraries, CC BY-NC unless otherwise noted. Notify us about copyright concerns.