FigurA11y: AI Assistance for Writing Scientific Alt Text
Name
3640543.3645212.pdf
Size
6.76 MB
Format
Adobe PDF
Checksum (MD5)
d5735a3d1df8765e48141e6ce1573d9e
Author(s) • •
Singh, Nikhil
Wang, Lucy Lu
Bragg, Jonathan
Date Issued
March 18, 2024
Publisher
ACM
Citation
Singh, Nikhil, Wang, Lucy Lu and Bragg, Jonathan. 2024. "FigurA11y: AI Assistance for Writing Scientific Alt Text."
Version
Final published version
Abstract
High-quality alt text is crucial for making scientific figures accessible to blind and low-vision readers. Crafting complete, accurate alt text is challenging even for domain experts, as published figures often depict complex visual information and readers have varied informational needs. These challenges, along with high diversity in figure types and domain-specific details, also limit the usefulness of fully automated approaches. Consequently, the prevalence of high-quality alt text is very low in scientific papers today. We investigate whether and how human-AI collaborative editing systems can help address the difficulty of writing high-quality alt text for complex scientific figures. We present FigurA11y, an interactive system that generates draft alt text and provides suggestions for author revisions using a pipeline driven by extracted figure and paper metadata. We test two versions, motivated by prior work on visual accessibility and writing support. The base Draft+Revise version provides authors with an automatically generated draft description to revise, along with extracted figure metadata and figure-specific alt text guidelines to support the revision process. The full Interactive Assistance version further adds contextualized suggestions: text snippets to iteratively produce descriptions, and hypothetical user questions with possible answers to reveal potential ambiguities and resolutions. In a study of authors (N=14), we found the system assisted them in efficiently producing descriptive alt text. Generated drafts and interface elements enabled authors to quickly initiate and edit detailed descriptions. Additionally, interactive suggestions from the full system prompted more iteration and highlighted aspects for authors to consider, resulting in greater deviation from the drafts without increased average cognitive load or manual effort.
Description
IUI '24: Proceedings of the 29th International Conference on Intelligent User Interfaces March 18–21, 2024, Greenville, SC, USA
MIT Department
Massachusetts Institute of Technology. Media Laboratory
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3640543.3645212