Video Enhancement with Task-Oriented Flow
Name
1711.09078.pdf
Description
Accepted version
Size
7.81 MB
Format
Adobe PDF
Checksum (MD5)
a6be9dc4fd5e6d0e1eb815a93ffe88e4
Author(s)
Xue, Tianfan
Chen, Baian
Wu, Jiajun
Wei, Donglai
Freeman, William T
Date Issued
2019
Journal
International Journal of Computer Vision
Publisher
Springer Science and Business Media LLC
Version
Author's final manuscript
Abstract
© 2019, Springer Science+Business Media, LLC, part of Springer Nature. Many video enhancement algorithms rely on optical flow to register frames in a video sequence. Precise flow estimation is however intractable; and optical flow itself is often a sub-optimal representation for particular video processing tasks. In this paper, we propose task-oriented flow (TOFlow), a motion representation learned in a self-supervised, task-specific manner. We design a neural network with a trainable motion estimation component and a video processing component, and train them jointly to learn the task-oriented flow. For evaluation, we build Vimeo-90K, a large-scale, high-quality video dataset for low-level video processing. TOFlow outperforms traditional optical flow on standard benchmarks as well as our Vimeo-90K dataset in three video processing tasks: frame interpolation, video denoising/deblocking, and video super-resolution.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1007/s11263-018-01144-2
