Umwelt: Accessible Structured Editing of Multi-Modal Data Representations
Name
3613904.3641996.pdf
Size
4.97 MB
Format
Adobe PDF
Checksum (MD5)
5e3983907dbae91edf28f3f163d290bc
Author(s)
Zong, Jonathan
Pedraza Pineros, Isabella
Chen, Mengzhu (Katie)
Hajas, Daniel
Satyanarayan, Arvind
Date Issued
May 11, 2024
Publisher
ACM
Citation
Zong, Jonathan, Pedraza Pineros, Isabella, Chen, Mengzhu (Katie), Hajas, Daniel and Satyanarayan, Arvind. 2024. "Umwelt: Accessible Structured Editing of Multi-Modal Data Representations."
Version
Final published version
Abstract
We present Umwelt, an authoring environment for interactive multimodal data representations. In contrast to prior approaches, which center the visual modality, Umwelt treats visualization, sonification, and textual description as coequal representations: they are all derived from a shared abstract data model, such that no modality is prioritized over the others. To simplify specification, Umwelt evaluates a set of heuristics to generate default multimodal representations that express a dataset’s functional relationships. To support smoothly moving between representations, Umwelt maintains a shared query predicated that is reified across all modalities — for instance, navigating the textual description also highlights the visualization and filters the sonification. In a study with 5 blind / low-vision expert users, we found that Umwelt’s multimodal representations afforded complementary overview and detailed perspectives on a dataset, allowing participants to fluidly shift between task- and representation-oriented ways of thinking.
Description
CHI '24: Proceedings of the CHI Conference on Human Factors in Computing Systems May 11–16, 2024, Honolulu, HI, USA
MIT Department
MIT Morningside Academy for Design
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3613904.3641996
