Balancing appearance and context in sketch interpretation
Author(s)Song, Yale; Davis, Randall; Ma, Kaichen; Penney, Dana L.
MetadataShow full item record
We describe a sketch interpretation system that detects and classifies clock numerals created by subjects taking the Clock Drawing Test, a clinical tool widely used to screen for cognitive impairments (e.g., dementia). We describe how it balances appearance and context, and document its performance on some 2,000 drawings (about 24K clock numerals) produced by a wide spectrum of patients. We calibrate the utility of different forms of context, describing experiments with Conditional Random Fields trained and tested using a variety of features. We identify context that contributes to interpreting otherwise ambiguous or incomprehensible strokes. We describe ST-slices, a novel representation that enables “unpeeling” the layers of ink that result when people overwrite, which often produces ink impossible to analyze if only the final drawing is examined. We characterize when ST-slices work, calibrate their impact on performance, and consider their breadth of applicability
DepartmentMassachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence
Association for the Advancement of Artificial Intelligence
Song, Yale, Randall Davis, Kaichen Ma and Dana L. Penney. "Balancing Appearance and Context in Sketch Interpretation." Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, New York, New York, USA 9–15 July 2016.
Author's final manuscript