Segmentation Based Tracking for Aerial Robot Global Localization in Unstructured Environments with Oblique Monocular Camera Orientation

Abstract

In the field of robotics, there has been a growing interest in multi-robot systems and their potential to improve the efficiency, scale, and reliability of tasks beyond what an individual robot can achieve. Global localization is a crucial task for autonomous robot navigation, specifically in the multi-agent scenario where robots need to localize within maps communicated by other agents. The scenario where vehicles are viewing their environments from the same perspective, or camera viewpoint, is well studied. However, when environments are mapped from different camera viewing angles, traditional methods fail to match visual features and thus fail to localize. The technical gap that this thesis addresses is when autonomous vehicles within a team are mapping the same environment from different viewpoints, specifically nadir and an oblique camera orientations in an unstructured environment. Many existing visual place recognition (VPR) methods fail to match visual features that look visually different due to appearance, illumination, or viewpoint changes and thus fail to localize. In this thesis, we demonstrate the shortcomings of previous work to generalize to an off-nadir camera angle and explore the benefits and challenges that arise with utilizing oblique imagery for visual feature detection and tracking. We propose a segmentation-based object tracking pipeline to improve tracking and environment mapping performance in this traditionally challenging scenario. Our approach consists of 1) a front-end auto-segmentation tracking pipeline followed by 2) a submap correspondence search, which exploits geometric consistencies between environment maps to align vehicle reference frames. We evaluate our approach on a challenging indoor, cluttered dataset and demonstrate a maximum precision 74% higher than traditional and learning-based baseline methods, with a map size 0.5% the size of the most memory conservative traditional baseline method.