Dynamic execution of temporal plans with sensing actions and bounded risk

Abstract

A special report on the cover of the June 2016 issue of the IEEE Spectrum magazine reads: "can we trust robots?" In a world that has been experiencing a seemingly irreversible process by which autonomous systems have been given increasingly more space in strategic areas such as transportation, manufacturing, energy supply, planetary exploration, and even medical surgeries, it is natural that we start asking ourselves if these systems could be held at the same or even higher levels of safety than we expect from humans. In an effort to make a contribution towards a world of autonomy that we can trust, this thesis argues that one necessary step in this direction is the endowment of autonomous agents with the ability to dynamically adapt to their environment while meeting strict safety guarantees. From a technical standpoint, we propose that autonomous agents in safety-critical applications be able to execute conditional plans (or policies) within risk bounds (also referred to as chance constraints). By being conditional, the plan allows the autonomous agent to adapt to its environment in real-time by conditioning the choice of activity to be executed on the agent's current level of knowledge, or belief, about the true state of world. This belief state is, in turn, a function of the history of potentially noisy sensor observations gathered by the agent from the environment. With respect to bounded risk, it refers to the fact that executing such conditional plans should guarantee to keep the agent "safe" - as defined by sets of state constraints - with high probability, while moving away from the conservatism of minimum risk approaches. In this thesis, we propose Chance-Constrained Partially Observable Markov Decision Processes (CC-POMDP's) as a formalism for conditional risk-bounded planning under uncertainty. Moreover, we present Risk-bounded AO* (RAO*), a heuristic forward search-based algorithm that searches for solutions to a CC-POMDP by leveraging admissible utility and risk heuristics to simultaneously guide the search and perform early pruning of overly-risky policy branches. In an effort to facilitate the specification of risk-bounded behavior by human modelers, we also present the Chance-constrained Reactive Model-based Programming Language (cRMPL), a novel variant of RMPL that incorporates chance constraints as part of its syntax. Finally, in support of the temporal planning applications with duration uncertainty that this thesis is concerned about, we present the Polynomial-time Algorithm for Risk-aware Scheduling (PARIS) and its extension to conditional scheduling of Probabilistic Temporal Plan Networks (PTPN's). The different tools and algorithms developed in the context of this thesis are combined to form the Conditional Planning for Autonomy with Risk (CLARK) system, a risk-aware conditional planning system that can generate chance-constrained, dynamic temporal plans for autonomous agents that must operate under uncertainty. With respect to our empirical validation, each component of CLARK is benchmarked against the relevant state of the art throughout the chapters, followed by several demonstrations of the whole CLARK system working in tandem with other building blocks of an architecture for autonomy.