Identifying Novel Emotions and Wellbeing of Horses from Videos Through Unsupervised Learning

Abstract

first_pageDownload PDFsettingsOrder Article Reprints Open AccessArticle Identifying Novel Emotions and Wellbeing of Horses from Videos Through Unsupervised Learning by Aarya Bhave 1ORCID,Emily Kieson 2ORCID,Alina Hafner 3 andPeter A. Gloor 1,*ORCID 1 Massachusetts Institute of Technology, System Design & Management, Cambridge, MA 02142, USA 2 Equine International, Cambridge CB22 5LD, UK 3 TUM School of Computation, Information and Technology, Technical University of Munich, Arcisstraße 21, 80333 Munich, Germany * Author to whom correspondence should be addressed. Sensors 2025, 25(3), 859; https://doi.org/10.3390/s25030859 Submission received: 5 January 2025 / Revised: 22 January 2025 / Accepted: 30 January 2025 / Published: 31 January 2025 (This article belongs to the Special Issue Emotion Recognition and Cognitive Behavior Analysis Based on Sensors) Downloadkeyboard_arrow_down Browse Figures Review Reports Versions Notes

Abstract This research applies unsupervised learning on a large original dataset of horses in the wild to identify previously unidentified horse emotions. We construct a novel, high-quality, diverse dataset of 3929 images consisting of five wild horse breeds worldwide at different geographical locations. We base our analysis on the seven Panksepp emotions of mammals “Exploring”, “Sadness”, “Playing”, “Rage”, “Fear”, “Affectionate” and “Lust”, along with one additional emotion “Pain” which has been shown to be highly relevant for horses. We apply the contrastive learning framework MoCo (Momentum Contrast for Unsupervised Visual Representation Learning) on our dataset to predict the seven Panksepp emotions and “Pain” using unsupervised learning. We significantly modify the MoCo framework, building a custom downstream classifier network that connects with a frozen CNN encoder that is pretrained using MoCo. Our method allows the encoder network to learn similarities and differences within image groups on its own without labels. The clusters thus formed are indicative of deeper nuances and complexities within a horse’s mood, which can possibly hint towards the existence of novel and complex equine emotions.