Video face replacement
Name
Matusik_Video face replacement.pdf
Size
915.6 KB
Format
Adobe PDF
Checksum (MD5)
7e15e7beec570b8c68bf3b9de128feeb
Author(s)
Dale, Kevin
Sunkavalli, Kalyan
Johnson, Micah K.
Vlasic, Daniel
Matusik, Wojciech
Pfister, Hanspeter
Date Issued
December 2011
Journal
Proceedings of the 2011 SIGGRAPH Asia Conference (SA '11)
Publisher
Association for Computing Machinery (ACM)
Citation
Kevin Dale, Kalyan Sunkavalli, Micah K. Johnson, Daniel Vlasic, Wojciech Matusik, and Hanspeter Pfister. 2011. Video face replacement. In Proceedings of the 2011 SIGGRAPH Asia Conference (SA '11). ACM, New York, NY, USA, , Article 130 , 10 pages.
Version
Author's final manuscript
Abstract
We present a method for replacing facial performances in video. Our approach accounts for differences in identity, visual appearance, speech, and timing between source and target videos. Unlike prior work, it does not require substantial manual operation or complex acquisition hardware, only single-camera video. We use a 3D multilinear model to track the facial performance in both videos. Using the corresponding 3D geometry, we warp the source to the target face and retime the source to match the target performance. We then compute an optimal seam through the video volume that maintains temporal consistency in the final composite. We showcase the use of our method on a variety of examples and present the result of a user study that suggests our results are difficult to distinguish from real video footage.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike 3.0
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/2024156.2024164
