Plug-and-Play Supervisory Control Using Muscle and Brain Signals for Real-Time Gesture and Error Detection
Author(s)M. Hasani, Ramin; H. Guenther, Frank; DelPreto, Joseph Jeff; Salazar Gomez, Andres Felipe; Gil, Stephanie; Rus, Daniela L; ... Show more Show less
MetadataShow full item record
Control of robots in safety-critical tasks and situations where costly errors may occur is paramount for realizing the vision of pervasive human-robot collaborations. For these cases, the ability to use human cognition in the loop can be key for recuperating safe robot operation. This paper combines two streams of human biosignals, electrical muscle and brain activity via EMG and EEG, respectively, to achieve fast and accurate human intervention in a supervisory control task. In particular, this paper presents an end-to-end system for continuous rolling-window classification of gestures that allows the human to actively correct the robot on demand, discrete classification of Error-Related Potential signals (unconsciously produced by the human supervisor’s brain when observing a robot error), and a framework that integrates these two classification streams for fast and effective human intervention. The system also allows “plug-and-play” operation, demonstrating accurate performance even with new users whose biosignals have not been used for training the classifiers. The resulting hybrid control system for safety-critical situations is evaluated with 7 untrained human subjects in a supervisory control scenario where an autonomous robot performs a multi-target selection task.
DepartmentMassachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory; Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Robotics: Science and Systems XIV
Robotics: Science and Systems Foundation
DelPreto, Joseph, et al. “Plug-and-Play Supervisory Control Using Muscle and Brain Signals for Real-Time Gesture and Error Detection.” Robotics: Science and Systems XIV, Robotics: Science and Systems Foundation, 2018.
Author's final manuscript