Robust end-to-end learning for autonomous vehicles

Abstract

Deep learning has been successfully applied to "end-to-end" learning of the autonomous driving task, where a deep neural network learns to predict steering control commands from camera data input. While these works support reactionary control, the representation learned is not usable for higher-level decision making required for autonomous navigation. This thesis tackles the problem of learning a representation to predict a continuous control probability distribution, and thus steering control options and bounds for those options, which can be used for autonomous navigation. Each mode in the learned distribution encodes a possible macro-action that the system could execute at that instant, and the covariances of the modes place bounds on safe steering control values. Our approach has the added advantage of being trained solely on unlabeled data collected from inexpensive cameras. In addition to uncertainty estimates computed directly by our model, we add robustness by developing a novel stochastic dropout sampling technique for estimating the inherent confidence of the model's output. We install the relevant processing hardware pipeline on-board a full-scale autonomous vehicle and integrate our learning algorithms for real-time control inference. Finally, we evaluate our models on a challenging dataset containing a wide variety of driving conditions, and show that the algorithms developed as part of this thesis are capable of successfully controlling the vehicle on real roads and even under a parallel autonomy paradigm wherein control is shared between human and robot.