Robust execution for stochastic hybrid systems

Abstract

Unmanned systems, such as Autonomous Underwater Vehicles (AUVs), planetary rovers and space probes, have enormous potential in areas such as reconnaissance and space exploration. However the effectiveness and robustness of these systems is currently restricted by a lack of autonomy. Previous work introduced the concept of a model-based executive, which increases the level of autonomy, elevating the level at which systems are commanded. This simplifies the operator's task and leaves degrees of freedom in the plan that allow the executive to optimize resources and ensure robustness to uncertainty. Uncertainty arises due to uncertain state estimation, disturbances, model uncertainty and component failures. This thesis develops a model-based executive that reasons explicitly from a stochastic hybrid discrete-continuous system model to find the optimal course of action, while ensuring the required level of robustness to uncertainty is achieved. Our first contribution is a novel 'Particle Control' approach for robust execution of state plans with stochastic hybrid systems. We introduce the notion of chance-constrained state plan execution; this means that the executive ensures tasks in the state plan have at least a specified minimum probability of success. The minimum probabilities are specified by the operator, enabling conservatism to be traded against performance. In order to make optimal chance-constrained execution tractable, the Particle Control approach approximates the system's state distribution using samples or 'particles' and optimizes the evolution of these particles to achieve chance-constrained state plan execution. In this manner particle control solves a tractable deterministic approximation to the original stochastic problem; furthermore the approximation becomes exact as the number of particles approaches infinity.

(cont.) For an important class of hybrid discrete-continuous system known as Jump Markov Linear Systems, the resulting deterministic optimization can be posed as a Mixed Integer Linear Program and solved to global optimality using efficient commercially-available solvers. Our second contribution is 'active' hybrid estimation subject to state plan constraints. Exact hybrid state estimation in stochastic hybrid systems is, in general, intractable. Tractable approximate hybrid estimation methods can lose track of the true hybrid state. In this thesis we develop an active hybrid estimation capability, which probes the system in order to reduce uncertainty in the hybrid state. This approach generates control sequences to minimize the probability of approximate hybrid estimation losing the true mode sequence, while ensuring that the state plan is satisfied subject to chance constraints. In order to make this problem tractable, we develop an analytic upper bound on the probability of losing the true mode sequence, and use a convex constraint tightening approach to approximate the chance constraints in the problem. Our final contribution is a novel hybrid model-learning approach. Specifying accurate hybrid system models is essential for accurate estimation and control, but is also extremely challenging. The hybrid executive must therefore determine hybrid system models from observed data. In this thesis we present an approximate Expectation-Maximization method for hybrid model learning; this method extends prior approaches to deal with mode transitions that depend on the continuous state.