Structuring Optimal Control of Legged Locomotion with Learning-based Methods

Abstract

Both optimal control methods and learning-based methods have been widely used for the control of legged locomotion. While optimal control formulations allow the designer to guarantee constraints on the solutions found, learning-based methods can leverage data and past experiences to globally search for solutions robust to noise and errors in the model parameters. This work explores how optimal control methods can be guided and structured by using data-driven techniques such as supervised learning and Bayesian optimization. Two case studies are presented. The first presents a model predictive controller for quadrupedal landing that reasons about body states, reaction forces, and contact timings in an online fashion. This highly nonlinear problem is made tractable by collecting thousands of feasible trajectories offline from trajectory optimizations and learning to generate them from the initial falling conditions. By initializing the search for a solution with the approximation from a deep neural network, the MIT Mini Cheetah is shown to be able to recover from significant falls in simulation and hardware in real time. The second studies the effect of learning command-dependent weights for a convex model predictive controller. The weights for the running costs are adjusted dynamically as a function of the command input. Using black-box optimization techniques and a defined higher-level reward, a function mapping the command input to the weights can be determined by sampling the trajectories from sweeps of command inputs.