Understanding and Predicting Image Memorability at a Large Scale
Name
Torralba_Understanding and predicting.pdf
Size
4.14 MB
Format
Adobe PDF
Checksum (MD5)
0ece725b4056be3b842fc028a31a0792
Author(s)
Khosla, Aditya
Raju, Akhil G.
Torralba, Antonio
Oliva, Aude
Date Issued
February 2016
Journal
2015 IEEE International Conference on Computer Vision (ICCV)
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Khosla, Aditya, et al. "Understanding and Predicting Image Memorability at a Large Scale." 2015 IEEE International Conference on Computer Vision (ICCV), 7-13 December 2015, Santiago, Chile, IEEE, 2015, pp. 2390–98.
Version
Author's final manuscript
Abstract
Progress in estimating visual memorability has been limited by the small scale and lack of variety of benchmark data. Here, we introduce a novel experimental procedure to objectively measure human memory, allowing us to build LaMem, the largest annotated image memorability dataset to date (containing 60,000 images from diverse sources). Using Convolutional Neural Networks (CNNs), we show that fine-tuned deep features outperform all other features by a large margin, reaching a rank correlation of 0.64, near human consistency (0.68). Analysis of the responses of the high-level CNN layers shows which objects and regions are positively, and negatively, correlated with memorability, allowing us to create memorability maps for each image and provide a concrete method to perform image memorability manipulation. This work demonstrates that one can now robustly estimate the memorability of images from many different classes, positioning memorability and deep memorability features as prime candidates to estimate the utility of information for cognitive systems. Our model and data are available at: http://memorability.csail.mit.edu.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology. Media Laboratory
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/ICCV.2015.275
