Predicting human behavior using visual media

Abstract

The ability to predict human behavior has applications in many domains ranging from advertising to education to medicine. In this thesis, I focus on the use of visual media such as images and videos to predict human behavior. Can we predict what images people remember or forget? Can we predict the type of images people will like? Can we use a photograph of someone to determine their state of mind? These are some of the questions I tackle in this thesis. Through my work, I demonstrate: (1) It is possible to predict with near human-level correlation, the probability with which people will remember images, (2) it is possible to predictably modify the extent to which a face photograph is remembered, (3) it is possible to predict, with a high correlation, the number of views an image will receive even before it is uploaded, (4) it is possible to accurately identify the gaze of people in images, both from the perspective of a device, and third-person. Further, I develop techniques to visualize and understand machine learning algorithms that could help humans better understand themselves through the analysis of algorithms capable of predicting behavior. Overall, I demonstrate that visual media is a rich resource for the prediction of human behavior.