Segment-based image matting using inpainting to resolve ambiguities

Abstract

Image Matting and Compositing [6, 25] - the extraction of a foreground element from an image and overlaying it over a different background - are two important operations in digital image manipulation. The extraction of the foreground element and its composition over an existing background is performed using a mask known as an alpha matte, which is generated by Image Matting. The problem of Image Matting is inherently ill-posed and has no "correct" solution; however, several matting algorithms have been proposed. This thesis studies the popular Bayesian Matting [9] algorithm in detail, and documents several problems with regard to its efficiency and accuracy. Inspired by these problems, this thesis proposes two major ideas: Firstly, a new Segment-Based Matting Algorithm that incorporates shading and has a closed form solution. Secondly, a new general approach that uses Digital Inpainting - the technique of restoring defective areas in digital images - to resolve ambiguous areas in the alpha mattes. This thesis demonstrates that the combination of these ideas improves both the efficiency and accuracy of Image Matting. From the results obtained, this thesis proposes the following idea: The degree of local smoothness enforced in the alpha matte should depend on the local color distribution; the more similar the local foreground and background color distributions are, the greater the amount of smoothness enforced.