G-VOILA: Gaze-Facilitated Information Querying in Daily Scenarios
Author(s)
Wang, Zeyu; Shi, Yuanchun; Wang, Yuntao; Yao, Yuchen; Yan, Kun; Wang, Yuhan; Ji, Lei; Xu, Xuhai; Yu, Chun; ... Show more Show less
Download3659623.pdf (5.547Mb)
Publisher with Creative Commons License
Publisher with Creative Commons License
Creative Commons Attribution
Terms of use
Metadata
Show full item recordAbstract
Modern information querying systems are progressively incorporating multimodal inputs like vision and audio. However, the integration of gaze --- a modality deeply linked to user intent and increasingly accessible via gaze-tracking wearables --- remains underexplored. This paper introduces a novel gaze-facilitated information querying paradigm, named G-VOILA, which synergizes users' gaze, visual field, and voice-based natural language queries to facilitate a more intuitive querying process. In a user-enactment study involving 21 participants in 3 daily scenarios (p = 21, scene = 3), we revealed the ambiguity in users' query language and a gaze-voice coordination pattern in users' natural query behaviors with G-VOILA. Based on the quantitative and qualitative findings, we developed a design framework for the G-VOILA paradigm, which effectively integrates the gaze data with the in-situ querying context. Then we implemented a G-VOILA proof-of-concept using cutting-edge deep learning techniques. A follow-up user study (p = 16, scene = 2) demonstrates its effectiveness by achieving both higher objective score and subjective score, compared to a baseline without gaze data. We further conducted interviews and provided insights for future gaze-facilitated information querying systems.
Date issued
2024-05-13Journal
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Publisher
Association for Computing Machinery
Citation
Wang, Zeyu, Shi, Yuanchun, Wang, Yuntao, Yao, Yuchen, Yan, Kun et al. 2024. "G-VOILA: Gaze-Facilitated Information Querying in Daily Scenarios." Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 8 (2).
Version: Final published version
ISSN
2474-9567
Collections
The following license files are associated with this item: