Use of a wearable camera system in conversation : toward a companion tool for social-emotional learning in autism

Abstract

Self-Cam is a wearable camera system that allows a person to collect video and audio from the movements of her own head and face. Like looking in a mirror, live feedback or video playback from the camera can be used to experience and learn how you look to others. Video playback and analysis tools can also be used to review and learn how others wearing the Self-Cam express themselves to you. We are developing a tool to teach the kind of facial analysis that an empathetic person might notice in interpretation of everyday interaction. Because it works from real life experience from a single person's point of view, we hypothesize that this analysis of self and immediate social environment will help the process of generalization of facial expression and mental state inference for that person, giving them a better understanding of the significance of facial movements and improvement in recognition of social cues. As steps toward this investigation, this thesis constructs a wearable camera system, designs a process of expression collection and analysis, and develops and implements a video test to evaluate the recognition abilities of study subjects throughout the investigation. Preliminary results show a great difference in ability between individual autistic subjects, some of whom approach the abilities of well scoring neurotypical individuals.