Sonora: Human-AI Co-Creation of 3D Audio Worlds and its Impact on Anxiety and Cognitive Load

Abstract

Soundscapes are widely used for relaxation, but their potential for personalized, navigable experiences remains under-explored. To address this, we developed Sonora, an AI tool that enables real-time generation of synthetic, spatialized soundscapes, allowing users to navigate immersive auditory environments and customize soundscapes using voice commands. Sonora’s architecture integrates audio diffusion models and LLMs within Unity3D. A between-subjects study with 32 participants investigated its effects on anxiety and user experience, compared to a control condition involving passive listening to a soundscape. Participants who interacted with Sonora reported higher entertainment than the control group. A positive correlation was found between state anxiety and user requests for Sonora, suggesting anxious users engaged more. Participants with moderate to high trait anxiety experienced significant reductions in state anxiety across both conditions, with no significant difference in cognitive load. Our findings highlight Sonora’s potential to promote relaxation, emphasizing the value of personalized experiences for mental health.