Hardware Implementation of a Complete Vision-Based Navigation Pipeline

Abstract

Autonomous navigation technology has made great advances, but many of the successful hardware systems are reliant on LiDAR. LiDAR is known to be expensive and to have high computation cost, while vision, which is typically used in combination with LiDAR, has additional benefits without the same cost concerns. It would be ideal if vision could replace LiDAR in its entirety, and there has been extensive work on vision-based alternatives for each module of the autonomy pipeline, but there are no well-established complete vision-based navigation pipelines. This project integrates vision-based object tracking, state estimation, and collision avoidance planning modules via the Robot Operating System and implements the system on hardware. Both the state estimation module, OpenVINS, and the object tracking module, CenterTrack 2D with depth images, are benchmarked on our hardware setup and found to have within 0.2 meters of displacement error. Trials of experiments in a real environment are performed to demonstrate the complete pipeline’s ability to navigate to a goal about 8 meters away in the presence of up to 6 naturally moving pedestrians.