Explanation Alignment: Quantifying the Correctness of
Model Reasoning At Scale
Name
bang-hbang-meng-eecs-2025-thesis.pdf
Description
Thesis PDF
Size
1.64 MB
Format
Adobe PDF
Checksum (MD5)
fafb931dd12b8273b2746c16ab81342d
Author(s)
Bang, Hyemin
Advisor(s)
Satyanarayan, Arvind
Date Issued
May 2025
Publisher
Massachusetts Institute of Technology
Abstract
To improve the reliability of machine learning models, researchers have developed metrics to measure the alignment between model saliency and human explanations. Thus far, however, these saliency-based alignment metrics have been used to conduct descriptive analyses and instance-level evaluations of models and saliency methods. To enable evaluative and comparative assessments of model alignment, we extend these metrics to compute explanation alignment — the aggregate agreement between model and human explanations. To compute explanation alignment, we aggregate saliency-based alignment metrics over many model decisions and report the result as a performance metric that quantifies how often model decisions are made for the right reasons. Through experiments on nearly 200 image classification models, multiple saliency methods, and MNIST, CelebA, and ImageNet tasks, we find that explanation alignment automatically identifies spurious correlations, such as model bias, and uncovers behavioral differences between nearly identical models. Further, we characterize the relationship between explanation alignment and model performance, evaluating the factors that impact explanation alignment and how to interpret its results in-practice.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
