Tracking engagement : a machine learning framework for estimating affective engagement

Abstract

Globally, construction fatality counts remain among the highest of all industries. As part of efforts to improve workers occupational health and safety, most companies provide workers with ongoing safety training. Yet accidents continue to take place, as there is a lack of understanding on how to increase the knowledge transfer that would help improve safety. The goal of this thesis is to automate and improve manual observation methods, presently used to determine construction workers' engagement during training courses by applying machine learning techniques to video images. This thesis proposes a framework to measure construction workers' engagement during training courses by unobtrusively analyzing engagement through body and pose estimation, codifying who is speaking and understating the predicted emotional state of a given worker through their facial expressions of emotion at specific lectures times through stateof- the-art computer vision techniques. The framework was prototyped on fifteen graduate and undergraduate students from a private university in the United States during four class sessions in a stadium set up classroom by three high definition cameras. The proposed system can enhance our understanding of learning processes within classroom contexts, while reducing the labor-intensive process of traditional observations methods, and allowing for the observation of a full class simultaneously. Further, the repeatability and standardization of objective observations will be improved as it will no longer depend on the skills of the observer and on his or her ability to capture and make sense of what was observed.