Zero-shot learning to execute tasks with robots

Abstract

This thesis explores multiple approaches for improving the state of the art in robotic planning with reinforcement learning. We are interested in designing a generalizable framework with several features, namely: allowing for zero-shot learning agents that are robust and resilient in the event of failing midway during a task, allowing us to detect failures, and being highly generalizable to new environments. Initially, we focused mostly on training agents that are resilient in the event of failure and robust to changing environments. For this, we first explore the use of deep Q networks to control a robot. Upon finding deep Q learning too unstable, we determine that Q networks alone are insufficient for attaining true resilience. Second, we explore the use of more powerful actor-critic methods, augmented with hindsight experience replay (HER). We determine that approaches requiring low-dimensional representations of the environment, such as HER, will not scale gracefully to handle more complex environments. Finally, we explore the use of generative models to learn a reward function that tightly couples the context of a linguistic command to the reward of a reinforcement learning agent. We hypothesize that a learned reward function will satisfy all of our criteria, and is part of our ongoing research.