Motion-Compensated Viewpoint Shift
Name
tao-juliust-meng-eecs-2024-thesis.pdf
Description
Thesis PDF
Size
39.39 MB
Format
Adobe PDF
Checksum (MD5)
285024859236b52583f9051809dc57bd
Author(s)
Tao, Julius L.
Advisor(s)
Lee, Chong U.
Lim, Jae S.
Date Issued
May 2024
Publisher
Massachusetts Institute of Technology
Abstract
Eye contact is an essential social cue that conveys our attention to others but is difficult to maintain during video calls. Many existing methods to synthesize a gaze-corrected view involve estimating a 3D face model and projecting it into the desired camera view, which is too computationally expensive for most personal computers. By drawing inspiration from 2D methods of video frame interpolation, we wish to not only correct eye gaze but also better align the face towards the camera without this expensive 3D modeling. Our findings suggest that adding a second webcam opposite the first and interpolating between the two outer camera views can give realistic, gaze-aligned center views. We conclude that the prevailing approach of 3D modeling is surprisingly not necessary for gaze correction. Not only do 2D techniques suffice, but their synthesized frames can appear more natural than prior results. We believe that this work is a crucial step towards true-to-life viewpoint shift for live video conferences.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
