Decision making in the presence of complex dynamics from limited, batch data

Abstract

Robot decision making in real-world domains can be extremely difficult when the robot has to interact with a complex, poorly understood environment. In these environments, a data-driven approach is commonly taken where a model is first learned and then used for decision making since expert knowledge is rarely sucient for specifying the world's dynamics. Unfortunately, learning a model for a complex environment often involves fitting a large number of parameters which can require an unobtainable amount of data. In real-world domains we are also typically confronted with fitting a model that is only an approximation of the true dynamics, causing difficulties for standard learning approaches. In this thesis we explore two core methodologies for learning a model for decision making in the presence of complex dynamics: explicitly selecting the model which achieves the highest estimated performance and allowing the model class to grow as more data is seen. We show that our approach for explicitly selecting the model with the highest estimated performance has desirable theoretical properties and outperforms standard minimum error fitting techniques on benchmark and real-world problems. To grow the size of model class with the amount of data, we first show how this can be accomplished by using Bayesian nonparametric statistics to model the dynamics, which can then be used for planning. We then present an alternative approach which grows the policy class using the principle of structural risk minimization, for which the resulting algorithm has provable performance bounds with weak assumptions on the true world's dynamics.