Face2Gesture: Translating Facial Expressions Into Robot Movements Through Shared Latent Space Neural Networks
Author(s)
Suguitan, Michael; DePalma, Nicholas; Hoffman, Guy; Hodgins, Jessica
Download3623386.pdf (3.263Mb)
Publisher Policy
Publisher Policy
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Terms of use
Metadata
Show full item recordAbstract
In this work, we present a method for personalizing human-robot interaction by using emotive facial expressions to generate affective robot movements. Movement is an important medium for robots to communicate affective states, but the expertise and time required to craft new robot movements promotes a reliance on fixed preprogrammed behaviors. Enabling robots to respond to multimodal user input with newly generated movements could stave off staleness of interaction and convey a deeper degree of affective understanding than current retrieval-based methods. We use autoencoder neural networks to compress robot movement data and facial expression images into a shared latent embedding space. Then, we use a reconstruction loss to generate movements from these embeddings and triplet loss to align the embeddings by emotion classes rather than data modality. To subjectively evaluate our method, we conducted a user survey and found that generated happy and sad movements could be matched to their source face images. However, angry movements were most often mismatched to sad images. This multimodal data-driven generative method can expand an interactive agent's behavior library and could be adopted for other multimodal affective applications.
Department
Massachusetts Institute of Technology. Media LaboratoryJournal
ACM Transactions on Human-Robot Interaction
Publisher
ACM
Citation
Suguitan, Michael, DePalma, Nicholas, Hoffman, Guy and Hodgins, Jessica. "Face2Gesture: Translating Facial Expressions Into Robot Movements Through Shared Latent Space Neural Networks." ACM Transactions on Human-Robot Interaction.
Version: Final published version
ISSN
2573-9522
Collections
The following license files are associated with this item: