2026-09-18T18:08:50Zhttps://dspace.mit.edu/server/oai/requestoai:dspace.mit.edu:1721.1/1569432024-09-25T03:30:11Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023
Katz, Boris
Cummings, Jesse E.
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
2024-09-24T18:22:03Z
2024-09-24T18:22:03Z
2024-05
2024-07-11T14:37:33.668Z
https://hdl.handle.net/1721.1/156943
In recent years, computational models trained to do object recognition have become increasingly capable. Models have demonstrated significant improvements and have achieved saturated performance on many standard image classification benchmarks sparking discussion of whether these models have achieved parity with human object recognition ability and whether we can consider this problem solved. However, these models continue to fail in real-world applications and in un-human-like ways creating a disparity between the performance that benchmarks report and the performance that users experience. In this thesis, we investigate why standard datasets are misaligned with real-world performance by exploring image recognition difficulty as defined by human psychophysics. Using behavioral experiments with humans, we are able to identify images that humans struggle to recognize and investigate the prevalence of these images in datasets and their effect on model performance. To shed light on how humans are able to recognize these images, we conduct preliminary analysis with neuroimaging to take the first steps at identifying the neural signature of image difficulty.
M.Eng.
Massachusetts Institute of Technology
Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)
Copyright retained by author(s)
https://creativecommons.org/licenses/by-nc-nd/4.0/
Characterizing Image Recognition Difficulty in Artificial and Biological Visual Processing
Thesis
application/pdf
Master
Master of Engineering in Electrical Engineering and Computer Science
Publication
unknown
unknown
<Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="9a9327a4-9a90-494d-bce9-1cf032d874ef">
<Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type>
<Title>Characterizing Image Recognition Difficulty in Artificial and Biological Visual Processing</Title>
<PublishedIn>
<Publication>
</Publication>
</PublishedIn>
<PublicationDate>2024-05</PublicationDate>
<Authors>
<Author>
<DisplayName>Cummings, Jesse E.</DisplayName>
<Affiliation>
<OrgUnit>
</OrgUnit>
</Affiliation>
</Author>
</Authors>
<Editors>
</Editors>
<Publishers>
<Publisher>
<DisplayName>Massachusetts Institute of Technology</DisplayName>
<OrgUnit />
</Publisher>
</Publishers>
<License>https://creativecommons.org/licenses/by-nc-nd/4.0/</License>
<Abstract>In recent years, computational models trained to do object recognition have become increasingly capable. Models have demonstrated significant improvements and have achieved saturated performance on many standard image classification benchmarks sparking discussion of whether these models have achieved parity with human object recognition ability and whether we can consider this problem solved. However, these models continue to fail in real-world applications and in un-human-like ways creating a disparity between the performance that benchmarks report and the performance that users experience. In this thesis, we investigate why standard datasets are misaligned with real-world performance by exploring image recognition difficulty as defined by human psychophysics. Using behavioral experiments with humans, we are able to identify images that humans struggle to recognize and investigate the prevalence of these images in datasets and their effect on model performance. To shed light on how humans are able to recognize these images, we conduct preliminary analysis with neuroimaging to take the first steps at identifying the neural signature of image difficulty.</Abstract>
<Access xmlns="http://purl.org/coar/access_right"
>
</Access>
</Publication>