Kimera: From SLAM to spatial perception with 3D dynamic scene graphs
Name
2101.06894.pdf
Description
Accepted version
Size
44.67 MB
Format
Adobe PDF
Checksum (MD5)
9afd827e82ed3b78e0a4e07865a67d7a
Author(s)
Rosinol, Antoni
Violette, Andrew
Abate, Marcus
Hughes, Nathan
Chang, Yun
Shi, Jingnan
Gupta, Arjun
Carlone, Luca
Date Issued
2021
Journal
International Journal of Robotics Research
Publisher
SAGE Publications
Citation
Rosinol, Antoni, Violette, Andrew, Abate, Marcus, Hughes, Nathan, Chang, Yun et al. 2021. "Kimera: From SLAM to spatial perception with 3D dynamic scene graphs." International Journal of Robotics Research, 40 (12-14).
Version
Author's final manuscript
Abstract
Humans are able to form a complex mental model of the environment they move in. This mental model captures geometric and semantic aspects of the scene, describes the environment at multiple levels of abstractions (e.g., objects, rooms, buildings), includes static and dynamic entities and their relations (e.g., a person is in a room at a given time). In contrast, current robots’ internal representations still provide a partial and fragmented understanding of the environment, either in the form of a sparse or dense set of geometric primitives (e.g., points, lines, planes, and voxels), or as a collection of objects. This article attempts to reduce the gap between robot and human perception by introducing a novel representation, a 3D dynamic scene graph (DSG), that seamlessly captures metric and semantic aspects of a dynamic environment. A DSG is a layered graph where nodes represent spatial concepts at different levels of abstraction, and edges represent spatiotemporal relations among nodes. Our second contribution is Kimera, the first fully automatic method to build a DSG from visual–inertial data. Kimera includes accurate algorithms for visual–inertial simultaneous localization and mapping (SLAM), metric–semantic 3D reconstruction, object localization, human pose and shape estimation, and scene parsing. Our third contribution is a comprehensive evaluation of Kimera in real-life datasets and photo-realistic simulations, including a newly released dataset, uHumans2, which simulates a collection of crowded indoor and outdoor scenes. Our evaluation shows that Kimera achieves competitive performance in visual–inertial SLAM, estimates an accurate 3D metric–semantic mesh model in real-time, and builds a DSG of a complex indoor environment with tens of objects and humans in minutes. Our final contribution is to showcase how to use a DSG for real-time hierarchical semantic path-planning. The core modules in Kimera have been released open source.
MIT Department
Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1177/02783649211056674
