3D model-based pose estimation of rigid objects from a single image for robotics

Abstract

We address the problem of finding the best 3D pose for a known object, supported on a horizontal plane, in a cluttered scene in which the object is not significantly occluded. We assume that we are operating with RGB-D images and some information about the pose of the camera. We also assume that a 3D mesh model of the object is available, along with a small number of labeled images of the object. The problem is motivated by robot systems operating in indoor environments that need to manipulate particular objects and therefore need accurate pose estimates. This contrasts with other vision settings in which there is great variability in the objects but precise localization is not required. Our approach is to find the global best object localization in a full 6D space of rigid poses. There are two key components to our approach: (1) learning a view-based model of the object and (2) detecting the object in an image. An object model consists of edge and depth parts whose positions are piece-wise linear functions of the object pose, learned from synthetic rendered images of the 3D mesh model. We search for objects using branch-and-bound search in the space of the depth image (not directly in the Euclidean world space) in order to facilitate an efficient bounding function computed from lower-dimensional data structures.