The Video Mesh: A Data Structure for Image-based Video Editing
Name
MIT-CSAIL-TR-2009-062.pdf
Size
8.55 MB
Format
Adobe PDF
Checksum (MD5)
f7d331644c7614f6fe871636f9f1eadd
Author(s)
Durand, Fredo
Cohen, Michael
Chen, Jiawen
Paris, Sylvain
Wang, Jue
Matusik, Wojciech
Advisor(s)
Fredo Durand
Date Issued
December 16, 2009
Citation
CHEN, J., PARIS, S., WANG, J., MATUSIK, W., COHEN, M., and DURAND, F. 2009. The Video Mesh: A Data Structure for Image-based Video Editing. MIT Computer Science and Artificial Intelligence Laboratory Technical Report Series, MIT-CSAIL-TR-2009-062.
Series/Report no.
MIT-CSAIL-TR-2009-062
Abstract
This paper introduces the video mesh, a data structure for representing video as 2.5D "paper cutouts." The video mesh allows interactive editing of moving objects and modeling of depth, which enables 3D effects and post-exposure camera control. The video mesh sparsely encodes optical flow as well as depth, and handles occlusion using local layering and alpha mattes. Motion is described by a sparse set of points tracked over time. Each point also stores a depth value. The video mesh is a triangulation over this point set and per-pixel information is obtained by interpolation. The user rotoscopes occluding contours and we introduce an algorithm to cut the video mesh along them. Object boundaries are refined with perpixel alpha values. The video mesh is at its core a set of texture mapped triangles, we leverage graphics hardware to enable interactive editing and rendering of a variety of effects. We demonstrate the effectiveness of our representation with a number of special effects including 3D viewpoint changes, object insertion, and depth-of-field manipulation.
Subjects
feature tracking
compositing
video editing
optical flow
triangle mesh
Terms of Use
Creative Commons Attribution-Noncommercial 3.0 Unported
Persistent DSpace Link
