Repository logo
Log in(current)
Repository logoMIT Open ScholarshipDSpace@MIT
  1. Home
  2. MIT Open Access Articles
  3. MIT Open Access Articles
  4. CultureNet: A Deep Learning Approach for Engagement Intensity Estimation from Face Images of Children with Autism

CultureNet: A Deep Learning Approach for Engagement Intensity Estimation from Face Images of Children with Autism

Thumbnail Image
Download
Name

iros2018_RudovicEtAl.pdf

Description
Accepted version
Size

3.8 MB

Format

Adobe PDF

Checksum (MD5)

47f1d3011199aaefa7d382b9480a6df5

sword-2019-07-31T18:43:24.original.xml (130 B)
Original SWORD entry document
Author(s)
Rudovic, Ognjen
•
Utsumi, Yuria
•
Lee, Jaeryoung
•
Hernandez, Javier
•
Ferrer, Eduardo Castello
•
Schuller, Bjorn
•
Picard, Rosalind W.
Date Issued
October 2018
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Rudovic, Ognjen, Utsumi, Yuria, Lee, Jaeryoung, Hernandez, Javier, Ferrer, Eduardo Castello et al. 2018. "CultureNet: A Deep Learning Approach for Engagement Intensity Estimation from Face Images of Children with Autism."
Version
Author's final manuscript
Abstract
© 2018 IEEE. Many children on autism spectrum have atypical behavioral expressions of engagement compared to their neu-rotypical peers. In this paper, we investigate the performance of deep learning models in the task of automated engagement estimation from face images of children with autism. Specifically, we use the video data of 30 children with different cultural backgrounds (Asia vs. Europe) recorded during a single session of a robot-assisted autism therapy. We perform a thorough evaluation of the proposed deep architectures for the target task, including within- and across-culture evaluations, as well as when using the child-independent and child-dependent settings. We also introduce a novel deep learning model, named CultureNet, which efficiently leverages the multi-cultural data when performing the adaptation of the proposed deep architecture to the target culture and child. We show that due to the highly heterogeneous nature of the image data of children with autism, the child-independent models lead to overall poor estimation of target engagement levels. On the other hand, when a small amount of data of target children is used to enhance the model learning, the estimation performance on the held-out data from those children increases significantly. This is the first time that the effects of individual and cultural differences in children with autism have empirically been studied in the context of deep learning performed directly from face images.
MIT Department
Massachusetts Institute of Technology. Media Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
http://creativecommons.org/licenses/by-nc-sa/4.0/
Persistent DSpace Link
https://hdl.handle.net/1721.1/137990
DOI of Published Version
https://doi.org/10.1109/iros.2018.8594177
Repository logo
PrivacyPermissionsAccessibilityContact us
Repository logo
Notify us about copyright concerns.