Toward robust deep stereo networks : uncertainty learning, novelty detection, and online adaptation

Abstract

Although deep learning continues to improve the state-of-the-art for stereo depth estimation on benchmark computer vision datasets, there are many remaining challenges in making these algorithms robust enough for deployment in real-world applications where reliability is critical [1, 2]. First, most deep stereo networks do not provide a measure of their own uncertainty, so we do not know when the network's depth estimates are reliable enough to use for other tasks such as planning, mapping, or localization [1]. Second, recent work has demonstrated that the accuracy of deep stereo networks can degrade considerably in novel environments that differ from those of the training set [3, 4, 5]. These failures due to domain shift make it difficult to guarantee performance in real-world applications where the visual environment may change unpredictably. On their own, existing deep stereo networks do not provide a principled way to detect and mitigate these domain shift effects [1]. This thesis makes three contributions towards creating deep stereo networks that are safer to deploy in novel environments. First, we experimentally demonstrate that training sets with predictable geometry encourage the network to learn brittle priors that do not generalize to new environments. We show that the network architecture and training set can be designed to minimize overfitting and improve generalization. Second, we train our deep stereo network to estimate aleatoric uncertainty. We demonstrate that the uncertainty predictions are well-calibrated with empirical error, and allow us to reason about the reliability of our model. Third, we show that the learned model of aleatoric uncertainty is useful for both detecting images from novel environments, and mitigating domain shift through online adaptation