Safe Nonlinear Control Under Control Constraints via Reachability, Optimal Control and Reinforcement Learning

Abstract

Autonomous robots in the real world have nonlinear dynamics with actuators that are subject to constraints. The combination of the two poses complicates the task of designing stabilizing controllers that can guarantee safety, which we denote as the stabilize-avoid problem. Existing control-based techniques can provide safety and stability guarantees but under the assumption of unbounded control inputs. On the other hand, learning-based techniques can handle control constraints but often are unable to correctly trade-off between safety and stability.

In this thesis, we take a step towards synthesizing controllers with improved safety and stability for high dimensional nonlinear systems with control constraints by combining techniques from reachability, optimal control, and reinforcement learning. We first propose a novel approach to solve constrained optimal control problems using deep reinforcement learning by using techniques from traditional constrained optimization, enabling the solution of stabilize-avoid problems for high-dimensional nonlinear systems with control constraints. Next, we present an alternate method of solving the stabilize-avoid problem using control barrier functions, where we present an improved method for learning control barrier functions for nonlinear systems with control constraints by drawing on connections between reachability and deep reinforcement learning.

We validate our proposed methods on a variety of benchmark tasks. Our experiments demonstrate the advantage of our methods over existing techniques in terms of improved safety rates and larger regions of attraction, especially in the case of high-dimensional systems.