Using a structured vocabulary to support machine understanding of student work

Abstract

This thesis presents a method for machine interpretation of visual representations, including those that are hand-drawn, created by students solving elementary math problems. This interpretation system extends a pen-based wireless classroom interaction system called Classroom Learning Partner. The key idea behind the interpretation is to employ a structured vocabulary that provides students with tools that give them enough structure to facilitate machine interpretation, but not so much that they cannot be creative in making their own representations. This structured vocabulary consists of images, shapes, tiles, and stamps and enables students to create visual representations that are constructed more easily and quickly than with freehand drawing. A machine can construct an interpretation of the visual representation by finding the relationships between the objects used in the representation, focusing on object type, location, or additional "digital ink" lines that indicate grouping of objects. The interpretation methods were evaluated on examples of student work collected in classroom trials in fourth grade classrooms in the Boston area. The results indicate that the interpretation methods will enable teachers to easily and quickly view student work in real time in a classroom and will provide teachers with information about their students' understanding of concepts underlying the visual representations.