Safe and robust reinforcement learning with neural network policies

Abstract

Autonomous vehicles operate in safety-critical environments, but heavily rely on seemingly black-box predictions from Deep Neural Networks. The danger of this reliance arises from the vulnerability of neural networks. Neural networks can fail overconfidently on novel data, i.e. test points that are far from the collection of training points. In addition, neural networks will also fail on test points that are close to training points, but crafted to fool the network, also called adversarial examples. The first framework developed in this thesis samples predictions from an ensemble of stochastic neural networks and detects novel data via high sample variance at test time. The gained sensitivity to novel data is embedded in a Safe Reinforcement Learning framework to achieve robustness to novelties in a particularly challenging safety-critical task, navigating a robot in pedestrian crowds. The resulting policy detects pedestrians that exhibit novel behavior at test time and avoids them cautiously. The second framework in this thesis is concerned with the failure of neural networks on adversarial examples, that have even been crafted to fool real-world autonomous vehicle perception algorithms. Only few existing algorithms for defending against adverseries provide formal guarantees. These methods are embedded as an add-on verification toll for existing Deep Reinforcement Learning algorithms to ensure robustness against sensor noise or adversarial examples during test time. The resulting policy is evaluated in a pedestrian navigation scenario on the robustness to sensor noise and is shown to reduce the number of collisions between agents compared to a non-robust policy. Finally, the thesis presents work on optimizing the sensor field of view of autonomous vehicle testbeds that operates in pedestrian environments.