Learning to infer and execute 3D shape programs
Name
1901.02875.pdf
Description
Accepted version
Size
3.52 MB
Format
Adobe PDF
Checksum (MD5)
f27f423f0b558f27385d51778c191beb
Author(s)
Tian, Yonglong
Luo, Andrew
Sun, Xingyuan
Ellis, Kevin
Freeman, William T
Tenenbaum, Joshua B
Wu, Jiajun
Date Issued
May 2019
Journal
ICLR 2019: International Conference on Learning Representations
Citation
Tian, Yonglong et al. "Learning to infer and execute 3D shape programs." ICLR 2019: 7th International Conference on Learning Representations, May 6-9, 2019, New Orleans, Louisiana: url https://openreview.net/forum?id=rylNH20qFQ ©2019 Author(s)
Version
Author's final manuscript
Abstract
Human perception of 3D shapes goes beyond reconstructing them as a set of points or a composition of geometric primitives: we also effortlessly understand higher-level shape structure such as the repetition and reflective symmetry of object parts. In contrast, recent advances in 3D shape sensing focus more on low-level geometry but less on these higher-level relationships. In this paper, we propose 3D shape programs, integrating bottom-up recognition systems with top-down, symbolic program structure to capture both low-level geometry and high-level structural priors for 3D shapes. Because there are no annotations of shape programs for real shapes, we develop neural modules that not only learn to infer 3D shape programs from raw, unannotated shapes, but also to execute these programs for shape reconstruction. After initial bootstrapping, our end-to-end differentiable model learns 3D shape programs by reconstructing shapes in a self-supervised manner. Experiments demonstrate that our model accurately infers and executes 3D shape programs for highly complex shapes from various categories. It can also be integrated with an image-to-shape module to infer 3D shape programs directly from an RGB image, leading to 3D shape reconstructions that are both more accurate and more physically plausible.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
