Self-supervised sparse-to-dense: Self-supervised depth completion from LiDAR and monocular camera
Name
1807.00275.pdf
Description
Submitted version
Size
7.33 MB
Format
Adobe PDF
Checksum (MD5)
e0154bd20c97044d3c387f7eb43c5dbe
Author(s)
Ma, Fangchang
Venturelli Cavalheiro, Guilherme.
Karaman, Sertac
Date Issued
May 2019
Journal
2019 International Conference on Robotics and Automation (ICRA)
Publisher
IEEE
Citation
Ma, Fangchang, Guilherme Venturelli Cavalheiro and Sertac Karaman. “Self-supervised sparse-to-dense: Self-supervised depth completion from LiDAR and monocular camera.” Paper presented at the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, 20-24 May 2019, IEEE © 2019 The Author(s)
Version
Original manuscript
Abstract
© 2019 IEEE. Depth completion, the technique of estimating a dense depth image from sparse depth measurements, has a variety of applications in robotics and autonomous driving. However, depth completion faces 3 main challenges: the irregularly spaced pattern in the sparse depth input, the difficulty in handling multiple sensor modalities (when color images are available), as well as the lack of dense, pixel-level ground truth depth labels for training. In this work, we address all these challenges. Specifically, we develop a deep regression model to learn a direct mapping from sparse depth (and color images) input to dense depth prediction. We also propose a self-supervised training framework that requires only sequences of color and sparse depth images, without the need for dense depth labels. Our experiments demonstrate that the self-supervised framework outperforms a number of existing solutions trained with semi-dense annotations. Furthermore, when trained with semi-dense annotations, our network attains state-of-the-art accuracy and is the winning approach on the KITTI depth completion benchmark² at the time of submission. Furthermore, the self-supervised framework outperforms a number of existing solutions trained with semi-dense annotations.
MIT Department
Massachusetts Institute of Technology. Department of Aeronautics and Astronautics
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/ICRA.2019.8793637
