CoMo : a whiteboard that converses about code

Abstract

Software engineers routinely solve problems by brainstorming at whiteboards. Among other modes, they communicate with speech and sketch. Unfortunately, the whiteboard plays the role of a passive medium. It serves only as a place to draw. But what if it could engage in the conversation, even to a limited degree? Ideally it would help guide the engineers to a solution by being an active participant in the conversation. This thesis presents an early version of that vision: CoMo, a whiteboard that converses about code. CoMo is capable of engaging its user in a constrained mixed-initiative symmetric-multimodal conversation about a data structure manipulation. When it understands the data structure, it uses a code synthesis system to generate functioning C code. It can successfully hold a limited conversation and synthesize code for 50 manipulations on 8 data structures. This thesis further presents findings from an observational user study that helped guide the interaction with CoMo. Finally, this thesis presents the mixed-initiative code-generation framework that CoMo implements to achieve its interaction, and the insights about having limited natural conversations about data structure manipulations that were gleaned while creating the framework.