Learning to see physics via visual de-animation
Name
6620-learning-to-see-physics-via-visual-de-animation.pdf
Description
Published version
Size
1.11 MB
Format
Adobe PDF
Checksum (MD5)
d5afd424c557103bc493161a63539144
Author(s)
Wu, Jiajun
Lu, Erika
Kohli, Pushmeet
Freeman, William T
Tenenbaum, Joshua B
Date Issued
December 2017
Journal
Advances in Neural Information Processing Systems 30 (NIPS 2017)
Publisher
Neural Information Processing Systems Foundation, Inc
Citation
Wu, Jiajun et al. "Learning to see physics via visual de-animation." Advances in Neural Information Processing Systems 30 (NIPS 2017), December 2017, Long Beach, California, Neural Information Processing Systems Foundation, Inc, December 2017. © 2017 Neural Information Processing Systems Foundation, Inc.
Version
Final published version
Abstract
We introduce a paradigm for understanding physical scenes without human annotations. At the core of our system is a physical world representation that is first recovered by a perception module and then utilized by physics and graphics engines. During training, the perception module and the generative models learn by visual de-animation - interpreting and reconstructing the visual information stream. During testing, the system first recovers the physical world state, and then uses the generative models for reasoning and future prediction. Even more so than forward simulation, inverting a physics or graphics engine is a computationally hard problem; we overcome this challenge by using a convolutional inversion network. Our system quickly recognizes the physical world state from appearance and motion cues, and has the flexibility to incorporate both differentiable and non-differentiable physics and graphics engines. We evaluate our system on both synthetic and real datasets involving multiple physical scenes, and demonstrate that our system performs well on both physical state estimation and reasoning problems. We further show that the knowledge learned on the synthetic dataset generalizes to constrained real images.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://papers.nips.cc/paper/6620-learning-to-see-physics-via-visual-de-animation
