Mdp Optimal Control under Temporal Logic Constraints
Name
Rus_MDP optimal control.pdf
Size
446.31 KB
Format
Adobe PDF
Checksum (MD5)
2066b1b1d13270dbc83e8b30d7bbbe19
Author(s)
Ding, Xu Chu
Smith, Stephen L.
Belta, Calin
Rus, Daniela L.
Date Issued
December 2011
Journal
50th IEEE Conference on Decision and Control and European Control Conference 2011 (CDC-ECC)
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Ding, Xu Chu et al. “MDP Optimal Control Under Temporal Logic Constraints.” 50th IEEE Conference on Decision and Control and European Control Conference 2011 (CDC-ECC). 532–538.
Version
Author's final manuscript
Abstract
In this paper, we develop a method to automatically generate a control policy for a dynamical system modeled as a Markov Decision Process (MDP). The control specification is given as a Linear Temporal Logic (LTL) formula over a set of propositions defined on the states of the MDP. We synthesize a control policy such that the MDP satisfies the given specification almost surely, if such a policy exists. In addition, we designate an “optimizing proposition” to be repeatedly satisfied, and we formulate a novel optimization criterion in terms of minimizing the expected cost in between satisfactions of this proposition. We propose a sufficient condition for a policy to be optimal, and develop a dynamic programming algorithm that synthesizes a policy that is optimal under some conditions, and sub-optimal otherwise. This problem is motivated by robotic applications requiring persistent tasks, such as environmental monitoring or data gathering, to be performed.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. School of Engineering
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike 3.0
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/CDC.2011.6161122
