Reflecting music through movement : a body-syntonic approach to playing [with] the piano

Abstract

This thesis introduces and examines methods for the capture and reproduction of music on the piano that maintain a tight coupling between the body, the sound, and the physical instrument. For expert musicians, the body plays an indispensable role in the physical act of playing and the understanding of both musical structure and expressivity. However, many music learning technologies mistakenly assume that playing music is "playing the notes" and neglect the role of the body in the development of the musical mind. Drawing from research in telepresence, tangible interfaces, and augmented reality, I propose to bring the body back into the picture, literally and metaphorically, by augmenting a digital player piano with projection mapping. My platform synchronizes dynamic imagery with the piano's moving keys and acoustic sounds. I here focus on two main projects: MirrorFugue and Andante. Inspired by reflections on the lacquered surfaces of a grand piano, MirrorFugue simulates the presence of a virtual pianist whose reflection is actually playing the physically moving keys. It encourages anyone to take the seat left empty at the piano, to feel in his or her own body how music is expressed through the body of the performer, and to play along. Andante presents music as miniature figures that appear to walk and dance on the piano keyboard, physically striking a key with each stop. It conveys the expressivity in rhythms and phrases as well as musical structures through the bodies and movements of the figures. Both installations are designed as immersive "sandboxes" for the playful exploration of musical ideas. Beyond my projects, this thesis explores the parallels between music learning and learning and large. I discuss the connections between theories of music learning (particularly Dalcroze Eurhythmics) with theories of general mental development (Jean Piaget, Jerome Bruner, Seymour Papert, and Marvin Minsky), as well as how strategies from music learning could inform the art of learning in general.