Connecting Deep Learning Models to the Human Brain
Name
subramaniam-vsub851-meng-eecs-2024-thesis.pdf
Description
Thesis PDF
Size
28.33 MB
Format
Adobe PDF
Checksum (MD5)
ff633db5a0aecb3a71d26ed13dd790d0
Author(s)
Subramaniam, Vighnesh
Advisor(s)
Katz, Boris
Date Issued
May 2024
Publisher
Massachusetts Institute of Technology
Abstract
In this thesis, we introduce innovative methodologies for connecting new deep learning models, particularly models that integrate vision and language with human brain processing. These models have shown remarkable advancements in tasks such as object recognition, scene classification, and language processing, achieving near-human accuracy in some cases. This raises intriguing questions about how closely the computations and geometric structure of these models mirror that of the human brain. Our method starts with measuring brain activity in response to vision and language stimuli and then exposes these stimuli to deep learning models to collect their internal activations. We analyze the similarity between these activations and brain activity using a specific representational distance metric. We focus on introducing statistical algorithms to assess whether one model is significantly more similar with the brain than another. Through our novel methodology, we assess whether there’s a more significant correlation between brain regions and multimodal models compared to unimodal ones. Our investigation reveals brain areas associated with vision-language integration and models of vision-language integration that are potentially most similar to the brain.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)
Copyright retained by author(s)
Persistent DSpace Link
