Measuring and modifying the intrinsic memorability of images

Abstract

this thesis, I developed and carried out a procedure to measure the memorability of an image by running hundreds of human-trials and making use of a custom designed image dataset, the Mem60k dataset. The large store of ground-truth memorability data enabled a variety of insights and applications. The data revealed information about what qualities (emotional content, aesthetic appeal, etc.) in an image make it memorable. Convolutional neural networks (CNNs) trained on the data could predict an image's relative memorability with high accuracy. CNNs could also generate memorability heat maps which pinpoint which parts of an image are memorable. Finally, with additional usage of a massive image database, I designed a pipeline that could modify the intrinsic memorability of an image. The performance of each application was tested and measured by running further human trials.