An incremental sampling-based algorithm for stochastic optimal control
Name
Huynh.Karaman.ea.ICRA12.pdf
Size
503.35 KB
Format
Adobe PDF
Checksum (MD5)
1f190b7004923047a89de293ed64e750
Author(s)
Huynh, Vu Anh
Karaman, Sertac
Frazzoli, Emilio
Date Issued
February 2016
Journal
The International Journal of Robotics Research
Publisher
SAGE Publications
Citation
Huynh, Vu Anh, et al. “An Incremental Sampling-Based Algorithm for Stochastic Optimal Control.” The International Journal of Robotics Research, vol. 35, no. 4, Apr. 2016, pp. 305–33.
Version
Author's final manuscript
Abstract
In this paper, we consider a class of continuous-time, continuous-space stochastic optimal control problems. Using the Markov chain approximation method and recent advances in sampling-based algorithms for deterministic path planning, we propose a novel algorithm called the incremental Markov Decision Process to incrementally compute control policies that approximate arbitrarily well an optimal policy in terms of the expected cost. The main idea behind the algorithm is to generate a sequence of finite discretizations of the original problem through random sampling of the state space. At each iteration, the discretized problem is a Markov Decision Process that serves as an incrementally refined model of the original problem. We show that with probability one, (i) the sequence of the optimal value functions for each of the discretized problems converges uniformly to the optimal value function of the original stochastic optimal control problem, and (ii) the original optimal value function can be computed efficiently in an incremental manner using asynchronous value iterations. Thus, the proposed algorithm provides an anytime approach to the computation of optimal control policies of the continuous problem. The effectiveness of the proposed approach is demonstrated on motion planning and control problems in cluttered environments in the presence of process noise. Keywords: Stochastic optimal control, dynamical systems, randomized methods, robotics
MIT Department
Massachusetts Institute of Technology. Department of Aeronautics and Astronautics
Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1177/0278364915616866
