Towards Reinforcement-Learning-based Robot Navigation with 3D Scene Graphs

Abstract

Applying Reinforcement Learning (RL) for autonomous navigation has enormous potential in several robotics applications, including search and rescue operations. RL circumvents the need to manually specify a control policy for navigation and allows capturing aspects that are difficult to describe without relying on learning, e.g., that survivors or objects of interest are more likely to be found in specific regions of the environment. This is relevant for navigation policies guiding autonomous exploration and object search. To improve the performance of RL models guiding autonomous agents, we use 3D Scene Graphs (3DSGs) as a map representation. Previous work has shown that RL policies based on offline 3DSGs produce promising results in simulation, and this work takes initial steps towards extending these findings to 3DSGs produced online by Hydra, a new spatial perception system that builds 3DSGs in real-time. The work also provides an initial integration of the RL policies previously trained and evaluated in simulation [1] on a Unitree A1 quadruped robot. While the results are too preliminary to be conclusive, the thesis takes several integration steps towards deploying scene-graph-based RL policies for navigation on real robots.