Intuitive Audio Interaction and Control in Multi-Source Environments

Abstract

In an increasingly noisy world, managing auditory focus is a persistent challenge. This thesis explores how embodied interactions—primarily head tracking, alongside experiments with gaze tracking, speech commands, and audio-visual segmentation—can enhance user control over complex auditory environments. By linking head orientation to volume adjustments, we investigated whether natural, instinctive movements could serve as intuitive, hands-free mechanisms for isolating and amplifying relevant sounds. User studies revealed that head tracking is effective in structured audio contexts, such as music, where distinct sources are easily separable. However, its utility diminishes in dense, overlapping conversations, highlighting the need for finer control mechanisms. While gaze and segmentation offer promising refinements, cognitive load and system responsiveness remain key challenges. These findings underscore that embodied audio interaction must be adaptive, content-aware, and seamlessly integrated with user intent.This research contributes to human-computer interaction by demonstrating both the potential and limitations of movement-based audio control. Future work should refine multimodal fusion, improve segmentation accuracy, and enhance accessibility to create systems that dynamically respond to users’ natural behaviors—reducing cognitive strain and enabling more fluid, user-centric auditory experiences.