Robust decision-making with model uncertainty in aerospace systems

Abstract

Actual performance of sequential decision-making problems can be extremely sensitive to errors in the models, and this research addressed the role of robustness in coping with this uncertainty. The first part of this thesis presents a computationally efficient sampling methodology, Dirichlet Sigma Points, for solving robust Markov Decision Processes with transition probability uncertainty. A Dirichlet prior is used to model the uncertainty in the transition probabilities. This approach uses the first two moments of the Dirichlet to generates samples of the uncertain probabilities and uses these samples to find the optimal robust policy. The Dirichlet Sigma Point method requires a much smaller number of samples than conventional Monte Carlo approaches, and is empirically demonstrated to be a very good approximation to the robust solution obtained with a very large number of samples. The second part of this thesis discusses the area of robust hybrid estimation. Model uncertainty in hybrid estimation can result in significant covariance mismatches and inefficient estimates. The specific problem of covariance underestimation is addressed, and a new robust estimator is developed that finds the largest covariance admissible within a prescribed uncertainty set. The robust estimator can be found by solving a small convex optimization problem in conjunction with Monte Carlo sampling, and reduces estimation errors in the presence of transition probability uncertainty. The Dirichlet Sigma Points are extended to this problem to reduce the computational requirements of the estimator. In the final part of the thesis, the Dirichlet Sigma Points are extended for real-time adaptation. Using insight from estimation theory, a modified version of the Dirichlet Sigma Points is presented that significantly improves the response time of classical estimators. The thesis is concluded with hardware implementation of these robust and adaptive algorithms on the RAVEN testbed, demonstrating their applicability to real-life UAV missions.