2026-09-20T10:02:39Zhttps://dspace.mit.edu/server/oai/requestoai:dspace.mit.edu:1721.1/1230082026-06-06T00:48:55Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023
David K. Gifford.
Carter, Brandon M. (Machine learning scientist)
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
2019-11-22T00:01:53Z
2019-11-22T00:01:53Z
2019
2019
https://hdl.handle.net/1721.1/123008
1127567462
This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.
Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2019
Cataloged from student-submitted PDF version of thesis.
Includes bibliographical references (pages 73-77).
Recent progress in machine learning has come at the cost of interpretability, earning the field a reputation of producing opaque, "black-box" models. While deep neural networks are often able to achieve superior predictive accuracy over traditional models, the functions and representations they learn are usually highly nonlinear and difficult to interpret. This lack of interpretability hinders adoption of deep learning methods in fields such as medicine where understanding why a model made a decision is crucial. Existing techniques for explaining the decisions by black-box models are often restricted to either a specific type of predictor or are undesirably sensitive to factors unrelated to the model's decision-making process. In this thesis, we propose sufficient input subsets, minimal subsets of input features whose values form the basis for a model's decision. Our technique can rationalize decisions made by a black-box function on individual inputs and can also explain the basis for misclassifications. Moreover, general principles that globally govern a model's decision-making can be revealed by searching for clusters of such input patterns across many data points. Our approach is conceptually straightforward, entirely model-agnostic, simply implemented using instance-wise backward selection, and able to produce more concise rationales than existing techniques. We demonstrate the utility of our interpretation method on various neural network models trained on text, genomic, and image data.
by Brandon M. Carter.
M.Eng.
M.Eng. Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science
77 pages
eng
Massachusetts Institute of Technology
MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission.
http://dspace.mit.edu/handle/1721.1/7582
Electrical Engineering and Computer Science.
Interpreting black-box models through sufficient input subsets
Thesis
application/pdf
2019-11-22T00:01:53Z
Publication
Master
EECS
unknown
unknown
<Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="1d4719db-29ac-4d94-95ac-52c11c692eee">
<Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type>
<Language>eng</Language>
<Title>Interpreting black-box models through sufficient input subsets</Title>
<PublishedIn>
<Publication>
</Publication>
</PublishedIn>
<PublicationDate>2019</PublicationDate>
<Authors>
<Author>
<DisplayName>Carter, Brandon M. (Machine learning scientist)</DisplayName>
<Affiliation>
<OrgUnit>
</OrgUnit>
</Affiliation>
</Author>
</Authors>
<Editors>
</Editors>
<Publishers>
<Publisher>
<DisplayName>Massachusetts Institute of Technology</DisplayName>
<OrgUnit />
</Publisher>
</Publishers>
<License>http://dspace.mit.edu/handle/1721.1/7582</License>
<Keyword>Electrical Engineering and Computer Science.</Keyword>
<Abstract>Recent progress in machine learning has come at the cost of interpretability, earning the field a reputation of producing opaque, "black-box" models. While deep neural networks are often able to achieve superior predictive accuracy over traditional models, the functions and representations they learn are usually highly nonlinear and difficult to interpret. This lack of interpretability hinders adoption of deep learning methods in fields such as medicine where understanding why a model made a decision is crucial. Existing techniques for explaining the decisions by black-box models are often restricted to either a specific type of predictor or are undesirably sensitive to factors unrelated to the model's decision-making process. In this thesis, we propose sufficient input subsets, minimal subsets of input features whose values form the basis for a model's decision. Our technique can rationalize decisions made by a black-box function on individual inputs and can also explain the basis for misclassifications. Moreover, general principles that globally govern a model's decision-making can be revealed by searching for clusters of such input patterns across many data points. Our approach is conceptually straightforward, entirely model-agnostic, simply implemented using instance-wise backward selection, and able to produce more concise rationales than existing techniques. We demonstrate the utility of our interpretation method on various neural network models trained on text, genomic, and image data.</Abstract>
<Access xmlns="http://purl.org/coar/access_right"
>
</Access>
</Publication>