Towards a multimodal Ouija Board for aircraft carrier deck operations

Abstract

In this thesis, I present DeckAssistant, a novel interface for planning aircraft carrier deck operations. DeckAssistant is an intelligent, multimodal aircraft carrier Ouija Board. DeckAssistant's design is inspired by traditional Ouija Boards, featuring a large digital tabletop display with a scale model of an aircraft carrier deck and aircraft. Users plan aircraft operations by using this interface to mock situations on deck. Unlike traditional Ouija Board technology, DeckAssistant is designed to contribute to the planning process. We enable DeckAssistant with a basic understanding of deck operations and add multimodal functionality to our system. Users manipulate our model of the deck by issuing commands using speech and gestures. The system responds to users with its own synthesized speech and graphics. The result is a conversation of sorts between users and the Ouija Board as they work together to accomplish tasks.