Affective social quest (ASQ) : teaching emotion recognition with interactive media & wireless expressive toys

Abstract

In this thesis, I investigate new ways to use affective computing and multimedia tools to augment a child's learning of emotional expression. I develop the hypothesis that these tools can be particularly useful to children with autism and their practitioners. I test the hypothesis by building a candidate research system that comprises a screen on which are shown emotionally charged animated movie clips, together with a set of stuffed dolls through which a child can interact with the movies. Each doll embodied an emotional expression: happy, angry, sad, and surprise. In operation, the test children are shown one of 200 emotive clips and they respond by touching the doll whose expression matches that of the clip. An online guide and registration system allows a therapist to control and monitor the interactions. Six volunteer test children used the system at the Dan Marino Center in Ft Lauderdale and their reactions were observed. This served as verification that a system that manipulated movies and haptic interfaces was feasible and second, such a system could augment and potentially automate some of the human-intensive, repetitive aspects of existing behavioral therapy techniques. All six children responded to and attended to the system, with five of them completing three one-hour day visits comprising multiple sessions. Some children showed improvement in their matching of emotions and one child demonstrated generalization in a home setting.