Synthesizing Images of Humans in Unseen Poses
Name
1978.pdf
Description
Accepted version
Size
10 MB
Format
Adobe PDF
Checksum (MD5)
305c972a763bb264e55b74aefc84ebef
Author(s)
Balakrishnan, Guha
Zhao, Amy
Dalca, Adrian V.
Durand, Fredo
Guttag, John
Date Issued
June 2018
Publisher
IEEE
Citation
Balakrishnan, Guha, Zhao, Amy, Dalca, Adrian V., Durand, Fredo and Guttag, John. 2018. "Synthesizing Images of Humans in Unseen Poses."
Version
Author's final manuscript
Abstract
© 2018 IEEE. We address the computational problem of novel human pose synthesis. Given an image of a person and a desired pose, we produce a depiction of that person in that pose, retaining the appearance of both the person and background. We present a modular generative neural network that synthesizes unseen poses using training pairs of images and poses taken from human action videos. Our network separates a scene into different body part and background layers, moves body parts to new locations and refines their appearances, and composites the new foreground with a hole-filled background. These subtasks, implemented with separate modules, are trained jointly using only a single target image as a supervised label. We use an adversarial discriminator to force our network to synthesize realistic details conditioned on pose. We demonstrate image synthesis results on three action classes: Golf, yoga/workouts and tennis, and show that our method produces accurate results within action classes as well as across action classes. Given a sequence of desired poses, we also produce coherent videos of actions.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/cvpr.2018.00870
