Text-Driven Movie Manipulation

Abstract

In this thesis, we designed a method for manipulating a character’s appearance through a textual description of their desired appearance. Our method consisted of training a video-specific neural net model with an existing architecture for extracting keypoints, manipulating a representative frame to fit the user’s textual description through latent optimization, and producing a new video in a single forward pass. Our method requires an artist to follow 3 simple steps to manipulate a video and has only two inputs: a representative frame of the character the user wants to edit and a textual description of the desired appearance manipulation. Compared to work-intensive special effects, our method enables quick and flexible experimentation.