Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks
Name
6552-visual-dynamics-probabilistic-future-frame-synthesis-via-cross-convolutional-networks.pdf
Description
Published version
Size
1.23 MB
Format
Adobe PDF
Checksum (MD5)
ab76d40e6ba776e090e433152f991d8d
Author(s)
Xue, Tianfan
Wu, Jiajun
Bouman, Katherine L.
Freeman, William T.
Date Issued
2016
Citation
Xue, Tianfan, Wu, Jiajun, Bouman, Katherine L. and Freeman, William T. 2016. "Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks."
Version
Final published version
Abstract
© 2016 NIPS Foundation - All Rights Reserved. We study the problem of synthesizing a number of likely future frames from a single input image. In contrast to traditional methods, which have tackled this problem in a deterministic or non-parametric way, we propose to model future frames in a probabilistic manner. Our probabilistic model makes it possible for us to sample and synthesize many possible future frames from a single input image. To synthesize realistic movement of objects, we propose a novel network structure, namely a Cross Convolutional Network; this network encodes image and motion information as feature maps and convolutional kernels, respectively. In experiments, our model performs well on synthetic data, such as 2D shapes and animated game sprites, as well as on real-world video frames. We also show that our model can be applied to visual analogy-making, and present an analysis of the learned network representations.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://papers.nips.cc/paper/6552-visual-dynamics-probabilistic-future-frame-synthesis-via-cross-convolutional-networks
