Detecting Human Memory Processes via Bio-Signals

Abstract

Bio signals, such as eye movement data, photoplethysmography (PPG), and electrodermal activity (EDA), can provide insight into various cognitive states. Previous work has shown that eye movements along with other bio-signals differ when viewing familiar versus unfamiliar faces. Signals such as heart rate (derived from PPG) and skin conductance (derived from EDA) have also been previously evaluated to have correlations with different states of memory. In this study, we collected simultaneous pupillary, PPG, and EDA signals while participants (n=32) transitioned between several cognitive states (learning, recognition, and recall). Using this data, we propose multi-modal, machine learning methods to predict and evaluate whether a user is in a cognitive state of learning, recognition, or recall. We will discuss the differences observed in the data between these cognitive states, as well as next steps and applications for this model.