Geometrically and Temporally Consistent Robot Perception

Abstract

Perception algorithms rely on the capability to reconcile an ideal mathematical model of an object with imperfect data. Despite having been investigated extensively, robot perception remains challenging with spurious outliers, limited training data, and computational constraints. To this end, we investigate geometric and temporal consistency for modeling and inference in perception problems.

In this dissertation, we begin our work with the development of an end-to-end neural network model for 6D object pose estimation. The model applies a 3D fully-convolutional network to extract geometric features and enforces pairwise consistency of features via spectral convolution on a compatibility graph. We then develop a graph-theoretic framework for the hypothesis pruning problem. Specifically, we provide a planted clique perspective which draws the connection between a statistical model and robust estimation. This perspective leads to the design of a learning heuristic that is efficient and generalizable. Finally, we leverage temporal consistency for the 6D pose tracking of unknown objects in a video sequence. Our algorithm estimates the optical flow to produce temporally stable motion propagation, and optimizes scene structures and 6D object pose jointly. Instead of directly propagating pose estimation using frame-to-frame correlation, we register a target object in a new frame with a globally consistent model of the scene. Our method is shown to be efficient and accurate for 6D pose tracking.