Improvisational interaction : a framework for structural exploration of media

Abstract

Whenever we use computers to interact with media, our experience is that of direct control, and the goal of our interactions is either artifact-production (the editor paradigm) or passive exploration (the browser paradigm). This thesis proposes an alternative: a model of media interaction based on the ideas of non-idiomatic improvisation that encourages active exploration of media and its structures. We argue that in order to facilitate this kind of exploration, (1) computational tools must actively participate in the creative process and (2) the interaction framework must allow structural exploration of media. This leads to our main claim: improvisation should be considered a valid and appropriate paradigm for media interaction. To this extent, we present a Cognitive Model of Improvisational action (CMIA) that integrates element-centric and process-centric (structural) modes of control into a single framework for media exploration. This model allows participants to switch their attention between compositional elements and structural processes.

(cont.) The model is argued to be particularly powerful in leading us to novel spaces for media creation and consumption. We follow by presenting the Emonic Environment (Implementation), an interactive system built on the principles of CMIA. We conclude by describing two studies (Scenarios a Experiments) that analyze the ways in which Emonic Environment affects how people interact and think about their interactions with digital media. These studies illustrate the potential of CMIA as a paradigm for interaction between humans and machines.