Urban scene parsing via low-rank texture patches

Abstract

Automatic 3-D reconstruction of city scenes from ground, aerial, and satellite imagery is a difficult problem that has seen active research for nearly two decades. The problem is difficult because many algorithms require salient areas in the image to be identified and segmented, a task that is typically done by humans. We propose a pipeline that detects these salient areas using low-rank texture patches. Areas in images such as building facades contain low-rank textures, which are an intrinsic property of the scene and invariant to viewpoint. The pipeline uses these low-rank patches to automatically rectify images and detect and segment out the patches with an energy-minimizing graph cut. The output is then further parameterized to provide useful data to existing 3-D reconstruction methods. The pipeline was evaluated on challenging test images from Microsoft Bing Maps oblique aerial photography and produced an 80% recall and precision with superb empirical results.