Inductive Biases in Learning Hierarchical Abstractions for Bipedal Locomotion

Abstract

Bipedal locomotion presents a complex challenge in the field of reinforcement learning (RL), due to the high dimensional state and action space. Hierarchical abstractions and inductive biases emerge as critical components in navigating this complexity, offering pathways for effective learning and adaptation in bipedal locomotion tasks. By leveraging hierarchical structures and inductive biases, RL controllers can distill the inherent complexity of bipedal locomotion into manageable components, facilitating more efficient learning and adaptation processes. This work explores hierarchical abstractions within the context of RL for bipedal locomotion. We investigate three distinct RL locomotion controllers: a baseline controller, an action space abstraction controller, and a novel Hierarchical RL (HRL) controller implemented on velocity tracking tasks. We assess the controllers across various RL metrics, including task performance, learning efficiency, stability, and human-likeness metrics derived from human locomotion studies. We quantify the effectiveness of hierarchical abstractions and inductive biases in enhancing locomotion task performance and aligning RL-generated behaviors with human locomotion patterns. The action space abstraction controller emerges with superior performance, and our investigation underscores the potential of HRL approaches, indicative of its ability to leverage hierarchical structures for optimized locomotion behaviors and highlights the importance of selecting appropriate and well-designed abstractions. By analyzing the role of hierarchical abstractions and inductive biases in bipedal RL, our study contributes to advancing the understanding and development of RL algorithms for bipedal locomotion, with implications for the design of more efficient and human-like locomotion behaviors in robotic systems.