This is not the latest version of this item. The latest version can be found here.
Multimodal Autoencoder: A Deep Learning Approach to Filling In Missing Sensor Data and Enabling Better Mood Prediction
Name
17.Jaques_autoencoder_ACII.pdf
Description
Accepted version
Size
836.48 KB
Format
Adobe PDF
Checksum (MD5)
360c7fde73301c2234c575787cac1e96
Author(s)
Jaques, Natasha
Taylor, Sara
Sano, Akane
Picard, Rosalind
Date Issued
October 2017
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Jaques, Natasha, Taylor, Sara, Sano, Akane and Picard, Rosalind. 2017. "Multimodal Autoencoder: A Deep Learning Approach to Filling In Missing Sensor Data and Enabling Better Mood Prediction."
Version
Author's final manuscript
Abstract
© 2017 IEEE. To accomplish forecasting of mood in real-world situations, affective computing systems need to collect and learn from multimodal data collected over weeks or months of daily use. Such systems are likely to encounter frequent data loss, e.g. when a phone loses location access, or when a sensor is recharging. Lost data can handicap classifiers trained with all modalities present in the data. This paper describes a new technique for handling missing multimodal data using a specialized denoising autoencoder: The Multimodal Autoencoder (MMAE). Empirical results from over 200 participants and 5500 days of data demonstrate that the MMAE is able to predict the feature values from multiple missing modalities more accurately than reconstruction methods such as principal components analysis (PCA). We discuss several practical benefits of the MMAE's encoding and show that it can provide robust mood prediction even when up to three quarters of the data sources are lost.
MIT Department
Massachusetts Institute of Technology. Media Laboratory
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
10.1109/acii.2017.8273601
