Automated posture analysis for detecting learner's affective state

Abstract

As means of improving the ability of the computer to respond in a way that facilitates a productive and enjoyable learning experience, this thesis proposes a system for the automated recognition and dynamical analysis of natural occurring postures when a child is working in a learning-computer situation. Specifically, an experiment was conducted with 10 children between 8 and 11 years old to elicit natural occurring behaviors during a learning-computer task. Two studies were carried out; the first study reveals that 9 natural occurring postures are frequently repeated during the children's experiment; the second one shows that three teachers could reliably recognize 5 affective states (high interest, interest, low interest, taking a break and boredom). Hence, a static posture recognition system that distinguishes the set of 9 postures was built. This system senses the postures using two matrices of pressure sensors mounted on the seat and back of a chair. The matrices capture the pressure body distribution of a person sitting on the chair. Using Gaussian Mixtures and feed-forward Neural Network algorithms, the system classifies the postures in real time. It achieves an overall accuracy of 87.6% when it is tested with children's postures that were not included in the training set. Also, the children's posture sequences were dynamically analyzed using a Hidden Markov Model for representing each of the 5 affective states found by the teachers. As a result, only the affective states of high interest, low interest, and taking a break were recognized with an overall accuracy of 87% when tested with new postures sequences coming from children included in the training set. In contrast, when the system was tested with posture sequences coming from two subjects that were not included in the training set, it had an overall accuracy of 76%.