Isis, Cabbage and Viper : new tools and strategies for designing responsive media

Abstract

Responsive media are media that can sense and react intelligently to factors like presentation conditions or equipment, audience identity or profile, direct interaction, history or anticipation of involvement, and so on. The emergence of modern computational devices and communication networks, with their power to manipulate media signals and objects, has spurred the development of new forms of responsive media. But a lack of a broad framework for understanding the characteristics of and motivations for these media has resulted in computer-based design tools that do not mirror the character of our projects and working practices and, in turn, compromise our ability to communicate effectively with these media. This research strives to build such a framework and use it as a foundation for a suite of three new tools that better reflect the multi-layered, multi-sensory, and multi-person nature of responsive media. Aiming to rethink the principles underlying one of the most primary building blocks in the design process, the centerpiece of this suite, Isis, is a new programming language tailored to serve as a basis for responsive media.

(cont.) The second tool, Cabbage, is an experiment in employing a purist form of case-based reasoning in a system for creating responsive graphical layouts. Lastly, Viper is a new tool for making responsive video programs that can re-edit themselves to adapt to different viewing situations. These tools seek to enable the development of complex and meaningful input-output relationships through simple, elegant interfaces that emphasize visibility, accessibility, and extensibility.