Sketch recognition for Course of Action diagrams

Abstract

This thesis describes a software program that recognizes hand-drawn Course of Action diagrams. User input is through sketching, or a combination of sketching and speech. Course of Action symbols are recognized incrementally, and the informal sketching input is replaced with formal images of the symbols. The system uses the LADDER shape definition language to represent the geometric properties of shapes, and is capable of recognizing 327 distinct Course of Action symbols. The Intermediate Feature Recognizer is used to recognize shapes of intermediate complexity and is capable of recognizing some shapes that cannot be described using LADDER definions. By detecting features of intermediate complexity, the system is capable of automatic error correction of some stroke segmentation errors and dealing with filled-in and multi-segment lines. The system is also able to recognize a combination of speech and sketching input of some information that can't easily be communicated through sketching alone. The system has a shape grammar to allow the sketch recognizer to conform to rules for creating Course of Action symbols. The system is also capable of "interpreting" the sketch - understanding the higher-level details of military units and actions that were sketched in the Course of Action diagram.