Human-computer perception : modeling visual perceptual attributes
Author(s)
Newman, Anelise P.
Download1192966728-MIT.pdf (38.22Mb)
Other Contributors
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.
Advisor
Aude Oliva.
Terms of use
Metadata
Show full item recordAbstract
Human perception provides clues as to which visual content is most crucial or engaging. Where people look indicates what they pay attention to and find relevant; what people remember is what the human brain deems to be worthy of preservation. Recently, Deep Neural Networks have made it possible to predict cognitive attributes like saliency and memorability from just an image or video, at the same time that advances in human-computer interaction and human cognition have made collecting human data more accessible than ever. In this work, we aim to reinforce the interplay between human perception and computational models. We develop new strategies for collecting perceptual data, build models that predict human responses to visual stimuli, and show how applications of these models can be used to prioritize content for human consumption. First, we develop a toolbox of web-based user interfaces for crowdsourcing attention data using only a laptop or mobile phone. Through experimentation and analysis, we show how to deploy these interfaces to collect attention data scalably and flexibly for a variety of use cases. Next, we use our toolbox to study a novel aspect of human attention, resulting in the first saliency model that is capable of producing multiple saliency heatmaps corresponding to different potential viewing durations. Finally, we turn our focus to memorability, by designing an online memory game to measure and predict how likely a person is to remember a video. Systems like these that are capable of modeling human perception can make intelligent decisions about what information to prioritize, create, enhance, and preserve.
Description
Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, May, 2020 Cataloged from the official PDF of thesis. Includes bibliographical references (pages 89-99).
Date issued
2020Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer SciencePublisher
Massachusetts Institute of Technology
Keywords
Electrical Engineering and Computer Science.