Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks
Name
Shah_Efficient model.pdf
Size
2.67 MB
Format
Adobe PDF
Checksum (MD5)
1e0c5bef6b98374134ff438b6e14a96c
Author(s)
Shah, Julie A
Nikolaidis, Stefanos
Ramakrishnan, Ramya
Gu, Keren
Date Issued
March 2015
Journal
Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction - HRI '15
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Nikolaidis, Stefanos et al. “Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks.” ACM Press, 2015. 189–196.
Version
Author's final manuscript
Abstract
We present a framework for automatically learning human user models from joint-action demonstrations that enables a robot to compute a robust policy for a collaborative task with a human. First, the demonstrated action sequences are clustered into different human types using an unsupervised learning algorithm. A reward function is then learned for each type through the employment of an inverse reinforcement learning algorithm. The learned model is then incorporated into a mixed-observability Markov decision process (MOMDP) formulation, wherein the human type is a partially observable variable. With this framework, we can infer online the human type of a new user that was not included in the training set, and can compute a policy for the robot that will be aligned to the preference of this user. In a human subject experiment (n=30), participants agreed more strongly that the robot anticipated their actions when working with a robot incorporating the proposed framework (p<0.01), compared to manually annotating robot actions. In trials where participants faced difficulty annotating the robot actions to complete the task, the proposed framework significantly improved team efficiency (p<0.01). The robot incorporating the framework was also found to be more responsive to human actions compared to policies computed using a hand-coded reward function by a domain expert (p<0.01). These results indicate that learning human user models from joint-action demonstrations and encoding them in a MOMDP formalism can support effective teaming in human-robot collaborative tasks.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Aeronautics and Astronautics
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/2696454.2696455
