Sensor(y) Landscapes : technologies for new perceptual sensibilities

Abstract

When we listen closely, there is a pervading sense that we could hear more if we could only focus a little more intently. Our own perceptual limits are a moving target that we cannot delineate and rarely reach. This dissertation introduces technologies that operate at that mysterious boundary. I envision sensor(y) landscapes, physical sites that meld distributed sensing and sensory perception to afford new perceptual sensibilities. Today's mainstream technologies are well designed for rapid consumption of information and linear, sequential action. A side effect of their effectiveness to task, however, is a loss of undirected, curiosity-driven exploration in the world. I propose alternative technologies that would extend perceptual presence, amplify attention, and leverage intuitions. My focus is on turning rich sensor data into compelling sensory input, and as such, a substantial component of my work involved deploying sensor infrastructure in beautiful places. My projects center on a wetland restoration site, called Tidmarsh, where environmental data are densely and continuously collected and streamed. Using sound and vibration as the medium and nature as the setting, I undertook this work in two steps. The first constructs environments suffused with sensing and built for being present in. My projects in this space comprise sensor-driven virtual worlds, glass elevator sound installations, and vibrating forests that give oral histories. Building on lessons and infrastructure from the first approach, my culminating work uses non-occluding spatial audio to create situated perceptions of data. I developed a bone-conduction headphone device, called HearThere, that renders a live soundscape from distributed microphones and sensors, fully merged with the user's natural hearing. HearThere combines its wearer's inferred listening state with classification output from an Al engine to adjust the mix and spatial parameters of virtual audio sources. The device was developed based on findings from lab studies into spatial hearing and attention, and evaluated in a human subjects study with a panel of experts. Through these projects, I found that deriving meaning in the medium is a matter of possessing or developing perceptual sensibilities, intuitions for how the permeated data can be teased out and contemplated. Carefully composed perceptual confusion-a blurring of place and distributed media-becomes an opportunity for the development of new transpresence sensibilities. How do users make sense of these new dimensions of perception, and how can technologies be designed to facilitate perceptual sense-making?