Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimization
Name
2201.13360.pdf
Description
Accepted version
Size
12.9 MB
Format
Adobe PDF
Checksum (MD5)
02707b74511901db8a7bdf112256b225
Author(s)
Hughes, Nathan
Chang, Yun
Carlone, Luca
Date Issued
2022
Journal
ROBOTICS: SCIENCE AND SYSTEM XVIII
Citation
Hughes, Nathan, Chang, Yun and Carlone, Luca. 2022. "Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimization." ROBOTICS: SCIENCE AND SYSTEM XVIII.
Version
Author's final manuscript
Abstract
3D scene graphs have recently emerged as a powerful high-level representation of 3D environments. A 3D scene graph models the environment as a layered graph where nodes represent spatial concepts at multiple levels of abstraction (from low-level geometry to high-level semantics including objects, places, rooms, buildings, etc.) and edges represent relations between concepts. While 3D scene graphs can serve as an advanced “mental model” for robots, how to build such a rich representation in real-time is still uncharted territory. This paper describes a real-time Spatial Perception System, a suite of algorithms to build a 3D scene graph from sensor data in real-time. Our first contribution is to develop real-time algorithms to incrementally construct the layers of a scene graph as the robot explores the environment; these algorithms build a local ESDF around the current robot trajectory estimate, extract a topological map of places from the ESDF, and then segment the places into rooms using an approach inspired by community-detection techniques. Our second contribution is to investigate loop closure detection and optimization in 3D scene graphs. We show that 3D scene graphs allow defining hierarchical descriptors for place recognition; our descriptors capture statistics across layers in the scene graph, ranging from low-level visual appearance, to summary statistics about objects and places. We then propose the first algorithm to optimize a 3D scene graph in response to loop closures; our approach relies on embedded deformation graphs to simultaneously correct all layers of the scene graph. We implement the proposed system into a highly parallelized architecture, named Hydra, that combines fast early and mid-level perception processes with slower high-level perception. We evaluate Hydra on simulated and real data and show it is able to reconstruct 3D scene graphs with an accuracy comparable with batch offline methods, while running online.
MIT Department
Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://roboticsconference.org/program/papers/050/
