Underwater Semantic Simultaneous Localization and Mapping

Abstract

Building semantically meaningful object level maps of underwater environments is crucial for enabling higher-level autonomy, fostering human-robot collaboration, and providing compressed map representations for bandwidth-constrained underwater communications, while localizing against such maps can improve the positioning accuracy of underwater vehicles by correcting for odometric drift. However, underwater semantic simultaneous localization and mapping (SLAM) has lagged behind analogous terrestrial and aerial semantic SLAM techniques largely due to the lack of large labeled underwater datasets and the challenging sensor modalities specific to underwater environments. To address these shortcomings, this thesis develops a range of methodologies to advance underwater semantic SLAM capabilities.

First, self-supervised learning and visual foundation models are leveraged to detect and segment underwater objects in an open-set manner, i.e., objects need not be present in the training dataset to be detected. The machinery of the open-set object detection technique breaks several assumptions made by existing closed-set semantic SLAM methods. Thus, new methods for object representation and data association are proposed and demonstrated. A method to localize underwater objects is then developed through an analysis of the geometry of underwater monocular cameras and multibeam sonars.

Finally, a formulation of open-set object-level place recognition as a graph matching problem is introduced. The formulation includes a method for calculating and tracking semantic uncertainty for open-set object detections. Experimental results on both underwater and terrestrial datasets demonstrate that the proposed formulation can be used for real-time accurate open-set object-based place recognition.

In summary, techniques for underwater object detection, localization, and data association are introduced and integrated with probabilistic graphical models for open-set semantic SLAM. The proposed techniques are tested across a wide variety of scenarios, and are shown to generalize to terrestrial settings as well.