SIFT Flow: Dense Correspondence across Scenes and its Applications
Name
Torralba_Sift flow.pdf
Size
9.47 MB
Format
Adobe PDF
Checksum (MD5)
8915cda13658306835a180ce598f4dc7
Author(s)
Liu, Ce
Yuen, Jenny
Torralba, Antonio
Date Issued
August 2010
Journal
IEEE transactions on pattern analysis and machine intelligence.
Publisher
Institute of Electrical and Electronics Engineers / IEEE Computer Society
Citation
Liu, Ce, Jenny Yuen, and Antonio Torralba. “SIFT Flow: Dense Correspondence across Scenes and Its Applications.” Pattern Analysis and Machine Intelligence, IEEE Transactions On 33.5 (2011) : 978-994. Copyright © 2011, IEEE
Version
Author's final manuscript
Abstract
While image alignment has been studied in different areas of computer vision for decades, aligning images depicting different scenes remains a challenging problem. Analogous to optical flow, where an image is aligned to its temporally adjacent frame, we propose SIFT flow, a method to align an image to its nearest neighbors in a large image corpus containing a variety of scenes. The SIFT flow algorithm consists of matching densely sampled, pixelwise SIFT features between two images while preserving spatial discontinuities. The SIFT features allow robust matching across different scene/object appearances, whereas the discontinuity-preserving spatial model allows matching of objects located at different parts of the scene. Experiments show that the proposed approach robustly aligns complex scene pairs containing significant spatial differences. Based on SIFT flow, we propose an alignment-based large database framework for image analysis and synthesis, where image information is transferred from the nearest neighbors to a query image according to the dense scene correspondence. This framework is demonstrated through concrete applications such as motion field prediction from a single image, motion synthesis via object transfer, satellite image registration, and face recognition.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike 3.0
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/TPAMI.2010.147
