3D reconstruction of cuboid-shaped objects from labeled images

Abstract

In this thesis, my goal is to determine a rectangular 3D cuboid that outlines the boundaries of a cuboid-shaped object shown in an image. The position of the corners of each cuboid are manually labeled, or annotated, in a 2D color image. Given the color image, the labels, and a 2D depth image of the same scene, an algorithm extrapolates a cuboid's 3D position, orientation, and size characteristics by minimizing two quantities: the deviation of each estimated corner's projected position from its annotated position in 2D space and the distance from each estimated surface to the observed points associated with that surface in 3D space. I found that this approach successfully estimated the 3D boundaries of a cuboid object for 72.6% of the cuboids in a data set of 1,089 manually-labeled cuboids in images taken from the SUN3D database.