Learning and Using the Arrow of Time
Name
Learning and Using the Arrow of Time (Supplementary Data).pdf
Description
Accepted version
Size
14.64 MB
Format
Adobe PDF
Checksum (MD5)
76442de62c75d4125b92732172ec127e
Author(s)
Wei, Donglai
Lim, Joseph
Zisserman, Andrew
Freeman, William T
Date Issued
June 2018
Publisher
IEEE
Citation
Wei, Donglai, Lim, Joseph, Zisserman, Andrew and Freeman, William T. 2018. "Learning and Using the Arrow of Time."
Version
Author's final manuscript
Abstract
© 2018 IEEE. We seek to understand the arrow of time in videos - what makes videos look like they are playing forwards or backwards? Can we visualize the cues? Can the arrow of time be a supervisory signal useful for activity analysis? To this end, we build three large-scale video datasets and apply a learning-based approach to these tasks. To learn the arrow of time efficiently and reliably, we design a ConvNet suitable for extended temporal footprints and for class activation visualization, and study the effect of artificial cues, such as cinematographic conventions, on learning. Our trained model achieves state-of-the-art performance on large-scale real-world video datasets. Through cluster analysis and localization of important regions for the prediction, we examine learned visual cues that are consistent among many samples and show when and where they occur. Lastly, we use the trained ConvNet for two applications: Relf-supervision for action recognition, and video forensics - determining whether Hollywood film clips have been deliberately reversed in time, often used as special effects.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/cvpr.2018.00840
