Learning static object segmentation from motion segmentation

Abstract

This thesis describes the SANE (Segmentation According to Natural Examples) algorithm for learning to segment objects in static images from video data. SANE uses background subtraction to find the segmentation of moving objects in videos. This provides object segmentation information for each video frame. The collection of frames and segmentations forms a training set that SANE uses to learn the image and shape properties that correspond to the observed motion boundaries. Then, when presented with new static images, the model infers segmentations similar to the observed motion segmentations. SANE is a general method for learning environment-specific segmentation models. Because it is self-supervised, it can adapt to a new environment and new objects with relative ease. Comparisons of its output to a leading image segmentation algorithm demonstrate that motion-defined object segmentation is a distinct problem from traditional image segmentation. The model outperforms a trained local boundary detector because it leverages the shape information it learned from the training data.