Bounds and Low-Rank Approximation for Controlled Markov Processes

Abstract

Stochastic processes have captivated scientific interest by balancing conceptual simplicity with the ability to model complex, poorly understood, or even entirely unknown phenomena. Still, the deployment of stochastic process models remains challenging in practice due to their intrinsically uncertain nature, complicating the computation and interpretation of model predictions. This thesis addresses two distinct challenges for the study of Markovian processes and associated control problems: certification and scale.

In the first part of this thesis, we present an algorithmic framework for conservatively answering the question: What is the best performance a controlled jump-diffusion process can attain? Answers to this question, even if conservative, shed light on fundamental limits, allowing us to distinguish situations where intrinsic noise masks poor decisions from situations where any attempt of improvement is futile. We connect infinite-dimensional linear programming over cones of occupation measures to techniques for approximating the solutions of Hamilton-Jacobi-Bellman and Kolmogorov backward equations. The result is a hierarchy of structured sum-of-squares programs that furnishes a sequence of hard, yet computable bounds for common control performance indicators encoding, for instance, operating cost or the probability of failure. These bounds in turn serve as witnesses of fundamental limitations or certificates of optimality and safety.

In the second part of this thesis, we explore the use of the framework developed in part one to shed light on the limits of quantum information technologies by quantifying the performance limits of controlled quantum devices. In the context of open-loop quantum control, our framework improves upon quantum speed limits, such as the Mandelstam-Tamm bound, and provably allows characterization of performance boundaries to arbitrary precision. For closed-loop controlled quantum systems, it constitutes the first ever approach to rigorously bound performance losses induced by continuous measurement.

The third part of the thesis is devoted to the challenge of scale as commonly encountered when studying Markov process models in high-dimensional spaces. Here, the complexity of computing and representing model predictions routinely exceeds available resources and renders interpretation challenging. We develop computational tools based on dynamical low-rank approximation that allow us to extract the dominant characteristic features of processes described by vast, nonlinear matrix-valued differential equations and track their evolution over time at a reduced cost.

The methods developed in this thesis are accompanied by software solutions exploiting the features of the Julia programming language to enable deployment of Markov process models in the context of scientific inquiry and engineering advancement more widely, with greater ease, and rigorous guarantees.