Symmetric multimodal interaction applied to database design and normalization

Abstract

A normal conversation between two people is typically multimodal, using both speech and gestures to effect communication. It is also symmetric because there is two-way multimodal interaction between the two parties. In contrast, when a human interacts with a computer, it is done through a strict and limited interface, usually a keyboard or mouse. Unlike the human-human conversation, this interaction is neither multimodal nor symmetric. The goal of this thesis is to empower computers to carry out symmetric, multimodal dialogues with humans, thereby providing a more natural human-computer interaction. To do so, we modified and extended Adler's Multimodal Interactive DialOgue System (MIDOS) to be a more flexible and domain-independent platform for supporting symmetric, multimodal interaction. We built an application that utilizes MIDOS in order to design and implement a normalized relational database, and then demonstrate the application's capabilities by using it to design the database for an after-action report wiki.