The Video Mesh: A Data Structure for Image-based Three-dimensional Video Editing
Name
Durand-The video mesh.pdf
Size
9.59 MB
Format
Adobe PDF
Checksum (MD5)
a4d8318b9e27dfd7fa54dd3d2bd251e6
Author(s)
Chen, Jiawen
Paris, Sylvain
Wang, Jue
Matusik, Wojciech
Cohen, Michael
Durand, Fredo
Date Issued
April 2011
Journal
2011 IEEE International Conference on Computational Photography (ICCP)
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Jiawen Chen et al. “The Video Mesh: A Data Structure for Image-based Three-dimensional Video Editing.” IEEE, 2011. 1–8. © Copyright 2011 IEEE
Version
Author's final manuscript
Abstract
This paper introduces the video mesh, a data structure for representing video as 2.5D “paper cutouts.” The video mesh allows interactive editing of moving objects and modeling of depth, which enables 3D effects and post-exposure camera control. The video mesh sparsely encodes optical flow as well as depth, and handles occlusion using local layering and alpha mattes. Motion is described by a sparse set of points tracked over time. Each point also stores a depth value. The video mesh is a triangulation over this point set and per-pixel information is obtained by interpolation. The user rotoscopes occluding contours and we introduce an algorithm to cut the video mesh along them. Object boundaries are refined with per-pixel alpha values. The video mesh is at its core a set of texture mapped triangles, we leverage graphics hardware to enable interactive editing and rendering of a variety of effects. We demonstrate the effectiveness of our representation with special effects such as 3D viewpoint changes, object insertion, depth-of-field manipulation, and 2D to 3D video conversion.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike 3.0
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/ICCPHOT.2011.5753118
