Generative Latent Motion Planning and Reinforcement Learning for Legged Locomotion

Abstract

In recent years, reinforcement learning has demonstrated its promise as a powerful tool for developing innovative and advanced control systems for legged robots. The method’s robustness, versatility, and generality have made it a prime candidate for future robotic systems deployed in the real world. Through the development of more advanced machine learning algorithms and more reliable and efficient physics simulators, reinforcement learning continues to improve and enable new, dynamic, and agile capabilities. While the results are often impressive and the tools relatively beginner-friendly, there remain impediments to scalable and reliable progress. Poor reward function scaling, challenges balancing exploration versus exploitation, and misalignment from the engineer’s intent are roadblocks to better performance. To get beyond these limitations, new tools and frameworks are necessary. In this work, I present novel methods to address these challenges and extend the capabilities of reinforcement learning on robot hardware. Through the quantification of the distributional sim-to-real gap, simulation model optimization for hardware matching, latent space motion sequence planning, and latent style training, I demonstrate never-before-seen performance on legged hardware.