Leveraging the Sensitivity of Plants with Deep Learning to Recognize Human Emotions
Name
sensors-24-01917-v2.pdf
Size
15.57 MB
Format
Adobe PDF
Checksum (MD5)
c9fd7a1f0c09572b726afe2678ee3c16
Author(s)
Kruse, Jakob Adrian
Ciechanowski, Leon
Dupuis, Ambre
Vazquez, Ignacio
Gloor, Peter A.
Date Issued
March 16, 2024
Journal
Sensors
Publisher
MDPI AG
Citation
Sensors 24 (6): 1917 (2024)
Version
Final published version
Abstract
Recent advances in artificial intelligence combined with behavioral sciences have led to the development of cutting-edge tools for recognizing human emotions based on text, video, audio, and physiological data. However, these data sources are expensive, intrusive, and regulated, unlike plants, which have been shown to be sensitive to human steps and sounds. A methodology to use plants as human emotion detectors is proposed. Electrical signals from plants were tracked and labeled based on video data. The labeled data were then used for classification., and the MLP, biLSTM, MFCC-CNN, MFCC-ResNet, Random Forest, 1-Dimensional CNN, and biLSTM (without windowing) models were set using a grid search algorithm with cross-validation. Finally, the best-parameterized models were trained and used on the test set for classification. The performance of this methodology was measured via a case study with 54 participants who were watching an emotionally charged video; as ground truth, their facial emotions were simultaneously measured using facial emotion analysis. The Random Forest model shows the best performance, particularly in recognizing high-arousal emotions, achieving an overall weighted accuracy of 55.2% and demonstrating high weighted recall in emotions such as fear (61.0%) and happiness (60.4%). The MFCC-ResNet model offers decently balanced results, with AccuracyMFCC−ResNet=0.318 and RecallMFCC−ResNet=0.324. Regarding the MFCC-ResNet model, fear and anger were recognized with 75% and 50% recall, respectively. Thus, using plants as an emotion recognition tool seems worth investigating, addressing both cost and privacy concerns.
Subjects
Electrical and Electronic Engineering
Biochemistry
Instrumentation
Atomic and Molecular Physics, and Optics
Analytical Chemistry
MIT Department
Massachusetts Institute of Technology. Center for Collective Intelligence
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.3390/s24061917
