BReach-LP: a Framework for Backward Reachability Analysis of Neural Feedback Loops

Abstract

Neural networks (NNs) can be used to solve a wide variety of robotics problems ranging from computer vision to control. However, while NNs often work well in nominal scenarios, their performance can decrease significantly in scenarios that they were not trained for. Thus, as we move toward real-world deployment of neural feedback loops (NFLs), i.e., closed-loop systems containing NNs, it is critical that we develop methods to verify that these systems are safe. Previous works have developed forward reachability techniques to verify safety for NFLs, but these techniques can be prohibitively conservative in non-convex settings such as obstacle avoidance. To enable safety verificaiton in non-convex settings, this thesis proposes BReach-LP: a set of techniques to conduct backward reachability analysis for NFLs. While backward reachability analysis has been studied for systems not containing NNs, the general noninvertability of NNs makes backward reachability analysis for NFLs a challenging problem. Thus, our approach leverages existing forward NN analysis tools to find affine bounds on the control inputs and solve a series of linear programs to efficiently find an approximation of the backprojection sets, i.e., the set of states for which an NN control policy will drive the system to a given target set. This thesis outlines four variations of BReach-LP, including proofs of their soundness and numerical results demonstrating their application.