Systems for Usable Machine Learning
Name
zytek-zyteka-phd-eecs-2025-thesis.pdf
Description
Thesis PDF
Size
11.55 MB
Format
Adobe PDF
Checksum (MD5)
d1182425c8d956f5854282450d0b8a4c
Author(s)
Zytek, Alexandra
Advisor(s)
Veeramachaneni, Kalyan
Date Issued
February 2025
Publisher
Massachusetts Institute of Technology
Abstract
Many real-world decision problems are complex, with outcomes difficult to measure and evaluate. The impact of decisions made in these domains is nuanced and takes a long time to be fully realized. Individual mistakes can lead to significant costs, and computational tools such as ML models must be integrated alongside existing, well-established human workflows. These properties of such decision problems means that ML solutions must be usable in order to be effective — in other words, developed and deployed in such a way as to be used by humans in decision-making and improve outcomes. In order improve ML usability, developers create ML tools, or diverse kinds of interfaces that allow users to understand ML models and their predictions. In this thesis, we use real-world case studies to synthesize generalizable lessons for applying usable ML tools to complex, real-world decision problems. Based on experience developing ML tools for child welfare screening, we propose a formal taxonomy of feature properties related to usability and interpretability. We then discuss the design and development of a system to make generating ML explanations that use such interpretable features more effective. Pyreal is a framework and Python library implementation that uses updated data transformers to generate explanations of ML models and predictions using interpretable features. Motivated by the development and customization effort required to develop ML tools for new applications, we then discuss the development of Sibyl, a configurable and comprehensive system for generating usable ML interfaces for a wide range of applications. We then discuss our case study in applying Sibyl to the decision problem of wind turbine monitoring. We then discuss Explingo, our system for transforming traditional ML explanations into natural language narratives to further improve the usability of ML outputs. We finish by discussing the practical lessons this work demonstrates related to the need for usable ML, the challenges specific to these complex applications, ethical questions, and future directions.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link