In Perfect Shape: Certifiably Optimal 3D Shape Reconstruction From 2D Landmarks
Name
1911.11924.pdf
Description
Accepted version
Size
25.92 MB
Format
Adobe PDF
Checksum (MD5)
5c911b373667f0ee1f3e0bf70ea64877
Author(s) •
Yang, Heng
Carlone, Luca
Date Issued
August 2020
Journal
Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Yang, Heng and Carlone, Luca. 2020. "In Perfect Shape: Certifiably Optimal 3D Shape Reconstruction From 2D Landmarks." Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.
Version
Author's final manuscript
Abstract
© 2020 IEEE We study the problem of 3D shape reconstruction from 2D landmarks extracted in a single image. We adopt the 3D deformable shape model and formulate the reconstruction as a joint optimization of the camera pose and the linear shape parameters. Our first contribution is to apply Lasserre's hierarchy of convex Sums-of-Squares (SOS) relaxations to solve the shape reconstruction problem and show that the SOS relaxation of minimum order 2 empirically solves the original non-convex problem exactly. Our second contribution is to exploit the structure of the polynomial in the objective function and find a reduced set of basis monomials for the SOS relaxation that significantly decreases the size of the resulting semidefinite program (SDP) without compromising its accuracy. These two contributions, to the best of our knowledge, lead to the first certifiably optimal solver for 3D shape reconstruction, that we name Shape#. Our third contribution is to add an outlier rejection layer to Shape# using a truncated least squares (TLS) robust cost function and leveraging graduated non-convexity to solve TLS without initialization. The result is a robust reconstruction algorithm, named Shape#, that tolerates a large amount of outlier measurements. We evaluate the performance of Shape and Shape# in both simulated and real experiments, showing that Shape# outperforms local optimization and previous convex relaxation techniques, while Shape# achieves state-of-the-art performance and is robust against 70% outliers in the FG3DCar dataset.
MIT Department
Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
Massachusetts Institute of Technology. Department of Aeronautics and Astronautics
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/CVPR42600.2020.00070