Regular Policies in Abstract Dynamic Programming
Name
16m1090946.pdf
Size
404.66 KB
Format
Adobe PDF
Checksum (MD5)
04220b72cb464fd58c77b7259c5bfa4c
Author(s)
Bertsekas, Dimitri P
Date Issued
January 2017
Journal
SIAM Journal on Optimization
Publisher
Society for Industrial & Applied Mathematics (SIAM)
Citation
Bertsekas, Dimitri P. “Regular Policies in Abstract Dynamic Programming.” SIAM Journal on Optimization 27, 3 (January 2017): 1694–1727 © 2017 Society for Industrial and Applied Mathematics
Version
Final published version
Abstract
We consider challenging dynamic programming models where the associated Bellman equation, and the value and policy iteration algorithms commonly exhibit complex and even pathological behavior. Our analysis is based on the new notion of regular policies. These are policies that are well-behaved with respect to value and policy iteration, and are patterned after proper policies, which are central in the theory of stochastic shortest path problems. We show that the optimal cost function over regular policies may have favorable value and policy iteration properties, which the optimal cost function over all policies need not have. We accordingly develop a unifying methodology to address long standing analytical and algorithmic issues in broad classes of undiscounted models, including stochastic and minimax shortest path problems, as well as positive cost, negative cost, risk-sensitive, and multiplicative cost problems.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1137/16M1090946
