Painterly interfaces for audiovisual performance

Abstract

This thesis presents a new computer interface metaphor for the real-time and simultaneous performance of dynamic imagery and sound. This metaphor is based on the idea of an inexhaustible, infinitely variable, time-based, audiovisual "substance" which can be gesturally created, deposited, manipulated and deleted in a free-form, non-diagrammatic image space. The interface metaphor is exemplified by five interactive audiovisual synthesis systems whose visual and aural dimensions are deeply plastic, commensurately malleable, and tightly connected by perceptually- motivated mappings. The principles, patterns and challenges which structured the design of these five software systems are extracted and discussed, after which the expressive capacities of the five systems are compared and evaluated.