Pose estimation using overhead imagery and semantics

Abstract

As hardware improves, so does the ability to run more complex algorithms. Improvements in the portability and performance of computing units has allowed for more compact robotic systems such as Micro Aerial Vehicles (MAVs), while developments in camera technologies has allowed for the generation of denser robot perception, such as depth and semantics, derived from environmental cues such as colour, shape and texture. Additionally, with the advent of commercial civilian satellites and initiatives like the Google Maps project, most of the earth's outdoor urban environments have been captured through satellite imagery. These overhead images contain useful information that can aid with outdoor robot navigation and localisation, by, for example, being used as prior maps in a localisation pipeline. This thesis presents a method for generating semantically consistent robot localisation updates on-board a MAV. The key contribution is a process that fuses local ground and global satellite image information through geometric alignment at the semantic level. This method uses a semantically segmented collapsed point cloud from a camera system on-board a MAV, without any notion of mapping, referencing only a potentially stale global overhead image. A particle filter is used to enable multi-hypothesis data association and allow for efficient recovery from motion uncertainty.