Where are they looking?
Name
965198467-MIT.pdf
Description
Full printable version
Size
6.24 MB
Format
Adobe PDF
Checksum (MD5)
f7c81383243e0c5a89624fae3c122952
Author(s)
Recasens, Adriá (Recasens Continente)
Advisor(s)
Antonio Torralba.
Date Issued
2016
Publisher
Massachusetts Institute of Technology
Abstract
Humans have the remarkable ability to follow the gaze of other people to identify what they are looking at. Following eye gaze, or gaze-following, is an important ability that allows us to understand what other people are thinking, the actions they are performing, and even predict what they might do next. Despite the importance of this topic, this problem has only been studied in limited scenarios within the computer vision community. In this thesis, we propose a deep neural network-based approach for gaze-following and a new benchmark dataset, GazeFollow, for thorough evaluation. Given an image and the location of a head, our approach follows the gaze of the person and identifies the object being looked at. Our deep network is able to discover how to extract head pose and gaze orientation, and to select objects in the scene that are in the predicted line of sight and likely to be looked at (such as televisions, balls and food). The quantitative evaluation shows that our approach produces reliable results, even when viewing only the back of the head. While our method outperforms several baseline approaches, we are still far from reaching human performance on this task. Furthermore, we show that gaze information is relevant to the action recognition problem. Overall, we believe that gaze-following is a challenging and important problem that deserves more attention from the community. Our model, code and dataset are available for download at http://gazefollow.csail.mit.edu
Description
Thesis: S.M. in Computer Science and Engineering, Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2016.
Cataloged from PDF version of thesis.
Includes bibliographical references (pages 53-55).
Subjects
Electrical Engineering and Computer Science.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission.
Persistent DSpace Link
