Show simple item record

dc.contributor.authorBorji, Ali
dc.contributor.authorBylinskii, Zoya
dc.contributor.authorRecasens Continente, Adria
dc.contributor.authorOliva, Aude
dc.contributor.authorTorralba, Antonio
dc.contributor.authorDurand, Frederic
dc.date.accessioned2018-01-30T15:39:06Z
dc.date.available2018-01-30T15:39:06Z
dc.date.issued2016-09
dc.identifier.isbn978-3-319-46453-4
dc.identifier.isbn978-3-319-46454-1
dc.identifier.issn0302-9743
dc.identifier.issn1611-3349
dc.identifier.urihttp://hdl.handle.net/1721.1/113344
dc.description.abstractRecently, large breakthroughs have been observed in saliency modeling. The top scores on saliency benchmarks have become dominated by neural network models of saliency, and some evaluation scores have begun to saturate. Large jumps in performance relative to previous models can be found across datasets, image types, and evaluation metrics. Have saliency models begun to converge on human performance? In this paper, we re-examine the current state-of-the-art using a fine-grained analysis on image types, individual images, and image regions. Using experiments to gather annotations for high-density regions of human eye fixations on images in two established saliency datasets, MIT300 and CAT2000, we quantify up to 60% of the remaining errors of saliency models. We argue that to continue to approach human-level performance, saliency models will need to discover higher-level concepts in images: text, objects of gaze and action, locations of motion, and expected locations of people in images. Moreover, they will need to reason about the relative importance of image regions, such as focusing on the most important person in the room or the most informative sign on the road. More accurately tracking performance will require finer-grained evaluations and metrics. Pushing performance further will require higher-level image understanding. Keywords: Saliency maps, Saliency estimation, Eye movements, Deep learning, Image understandingen_US
dc.description.sponsorshipNatural Sciences and Engineering Research Council of Canada (Postgraduate Scholarships-Doctoral Fellowship)en_US
dc.description.sponsorshipFundación Obra Social de La Caixa (Fellowship)en_US
dc.description.sponsorshipNational Science Foundation (U.S.) (Grant 1524817)en_US
dc.description.sponsorshipToyota Motor Corporation (Grant)en_US
dc.language.isoen_US
dc.publisherSpringeren_US
dc.relation.isversionofhttp://dx.doi.org/10.1007/978-3-319-46454-1_49en_US
dc.rightsCreative Commons Attribution-Noncommercial-Share Alikeen_US
dc.rights.urihttp://creativecommons.org/licenses/by-nc-sa/4.0/en_US
dc.sourceMIT Web Domainen_US
dc.titleWhere Should Saliency Models Look Next?en_US
dc.typeArticleen_US
dc.identifier.citationBylinskii, Zoya, et al. “Where Should Saliency Models Look Next?” Computer Vision – ECCV 2016, edited by Bastian Leibe et al., vol. 9909, Springer International Publishing, 2016, pp. 809–24.en_US
dc.contributor.departmentMassachusetts Institute of Technology. Department of Electrical Engineering and Computer Scienceen_US
dc.contributor.mitauthorBylinskii, Zoya
dc.contributor.mitauthorRecasens Continente, Adria
dc.contributor.mitauthorOliva, Aude
dc.contributor.mitauthorTorralba, Antonio
dc.contributor.mitauthorDurand, Frederic
dc.relation.journalEuroprean Conference on Computer Vision – ECCV 2016en_US
dc.eprint.versionAuthor's final manuscripten_US
dc.type.urihttp://purl.org/eprint/type/ConferencePaperen_US
eprint.statushttp://purl.org/eprint/status/NonPeerRevieweden_US
dspace.orderedauthorsBylinskii, Zoya; Recasens, Adrià; Borji, Ali; Oliva, Aude; Torralba, Antonio; Durand, Frédoen_US
dspace.embargo.termsNen_US
dc.identifier.orcidhttps://orcid.org/0000-0003-0941-9863
dc.identifier.orcidhttps://orcid.org/0000-0003-4915-0256
dc.identifier.orcidhttps://orcid.org/0000-0001-9919-069X
mit.licenseOPEN_ACCESS_POLICYen_US


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record