The body parametric : abstraction of vocal and physical expression in performance

Abstract

Performing artists have frequently used technology to sense and extend the body's natural expressivity via live control of multimedia. However, the sophistication, emotional content, and variety of expression possible through the original physical channels of voice and movement are generally not captured or represented by these technologies and thus cannot be intuitively transferred from body to digital media. Additionally, relevant components of expression vary between different artists, performance pieces, and output modalities, such that any single model for describing movement and the voice cannot be meaningful in all contexts. This dissertation presents a new framework for flexible parametric abstraction of expression in vocal and physical performance, the Expressive Performance Extension Framework. This framework includes a set of questions and principles to guide the development of new extended performance works and systems for performance extension, particularly those incorporating machine learning techniques. Second, this dissertation outlines the design of a multi-layered computational workflow that uses machine learning for the analysis and recognition of expressive qualities of movement and voice. Third, it introduces a performance extension toolkit, the Expressive Performance Extension System, that integrates key aspects of the theoretical framework and computational workflow into live performance contexts. This system and these methodologies have been tested through the creation of three performance and installation works: a public installation extending expressive physical movement (the Powers Sensor Chair), an installation exploring the expressive voice (Vocal Vibrations), and a set of performances extending the voice and body (Crenulations and Excursions and Temporal Excursions). This work lays the groundwork for systems that can be true extensions of and complements to a live performance, by recognizing and responding to subtleties of timing, articulation, and expression that make each performance fundamentally unrepeatable and unique.