Learning-Based Video Motion Magnification
Name
deepmag.pdf
Description
Accepted version
Size
10.91 MB
Format
Adobe PDF
Checksum (MD5)
83fa94e4e4e730269ef1c9453c7ab144
Author(s)
Oh, Tae-Hyun
Jaroensri, Ronnachai
Kim, Changil
Elgharib, Mohamed
Durand, Frédo
Freeman, William T.
Matusik, Wojciech
Date Issued
2018
Publisher
Springer International Publishing
Citation
Oh, Tae-Hyun, Jaroensri, Ronnachai, Kim, Changil, Elgharib, Mohamed, Durand, Frédo et al. 2018. "Learning-Based Video Motion Magnification."
Version
Author's final manuscript
Abstract
© 2018, Springer Nature Switzerland AG. Video motion magnification techniques allow us to see small motions previously invisible to the naked eyes, such as those of vibrating airplane wings, or swaying buildings under the influence of the wind. Because the motion is small, the magnification results are prone to noise or excessive blurring. The state of the art relies on hand-designed filters to extract representations that may not be optimal. In this paper, we seek to learn the filters directly from examples using deep convolutional neural networks. To make training tractable, we carefully design a synthetic dataset that captures small motion well, and use two-frame input for training. We show that the learned filters achieve high-quality results on real videos, with less ringing artifacts and better noise characteristics than previous methods. While our model is not trained with temporal filters, we found that the temporal filters can be used with our extracted representations up to a moderate magnification, enabling a frequency-based motion selection. Finally, we analyze the learned filters and show that they behave similarly to the derivative filters used in previous works. Our code, trained model, and datasets will be available online.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1007/978-3-030-01225-0_39
