CultureNet: A Deep Learning Approach for Engagement Intensity Estimation from Face Images of Children with Autism
Name
iros2018_RudovicEtAl.pdf
Description
Accepted version
Size
3.8 MB
Format
Adobe PDF
Checksum (MD5)
47f1d3011199aaefa7d382b9480a6df5
Author(s)
Rudovic, Ognjen
Utsumi, Yuria
Lee, Jaeryoung
Hernandez, Javier
Ferrer, Eduardo Castello
Schuller, Bjorn
Picard, Rosalind W.
Date Issued
October 2018
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Rudovic, Ognjen, Utsumi, Yuria, Lee, Jaeryoung, Hernandez, Javier, Ferrer, Eduardo Castello et al. 2018. "CultureNet: A Deep Learning Approach for Engagement Intensity Estimation from Face Images of Children with Autism."
Version
Author's final manuscript
Abstract
© 2018 IEEE. Many children on autism spectrum have atypical behavioral expressions of engagement compared to their neu-rotypical peers. In this paper, we investigate the performance of deep learning models in the task of automated engagement estimation from face images of children with autism. Specifically, we use the video data of 30 children with different cultural backgrounds (Asia vs. Europe) recorded during a single session of a robot-assisted autism therapy. We perform a thorough evaluation of the proposed deep architectures for the target task, including within- and across-culture evaluations, as well as when using the child-independent and child-dependent settings. We also introduce a novel deep learning model, named CultureNet, which efficiently leverages the multi-cultural data when performing the adaptation of the proposed deep architecture to the target culture and child. We show that due to the highly heterogeneous nature of the image data of children with autism, the child-independent models lead to overall poor estimation of target engagement levels. On the other hand, when a small amount of data of target children is used to enhance the model learning, the estimation performance on the held-out data from those children increases significantly. This is the first time that the effects of individual and cultural differences in children with autism have empirically been studied in the context of deep learning performed directly from face images.
MIT Department
Massachusetts Institute of Technology. Media Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/iros.2018.8594177
