MIT Libraries logoDSpace@MIT

MIT
View Item 
  • DSpace@MIT Home
  • MIT Open Access Articles
  • MIT Open Access Articles
  • View Item
  • DSpace@MIT Home
  • MIT Open Access Articles
  • MIT Open Access Articles
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

Approximate dynamic programming using model-free Bellman Residual Elimination

Author(s)
Bethke, Brett M.; How, Jonathan P.
Thumbnail
DownloadHow-2010-Approximate dynamic programming using model-free Bellman Residual Elimination.pdf (296.7Kb)
PUBLISHER_POLICY

Publisher Policy

Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.

Terms of use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Metadata
Show full item record
Abstract
This paper presents an modification to the method of Bellman Residual Elimination (BRE) for approximate dynamic programming. While prior work on BRE has focused on learning an approximate policy for an underlying Markov Decision Process (MDP) when the state transition model of the MDP is known, this work proposes a model-free variant of BRE that does not require knowledge of the state transition model. Instead, state trajectories of the system, generated using simulation and/or observations of the real system in operation, are used to build stochastic approximations of the quantities needed to carry out the BRE algorithm. The resulting algorithm can be shown to converge to the policy produced by the nominal, model-based BRE algorithm in the limit of observing an infinite number of trajectories. To validate the performance of the approach, we compare model-based and model-free BRE against LSPI, a well-known approximate dynamic programming algorithm. Measuring performance in terms of both computational complexity and policy quality, we present results showing that BRE performs at least as well as, and sometimes significantly better than, LSPI on a standard benchmark problem.
Date issued
2010-06
URI
http://hdl.handle.net/1721.1/66203
Department
Massachusetts Institute of Technology. Aerospace Controls Laboratory; Massachusetts Institute of Technology. Department of Aeronautics and Astronautics
Journal
American Control Conference 2010
Publisher
Institute of Electrical and Electronics Engineers / American Automatic Control Council
Citation
Bethke, B., and J.P. How. “Approximate dynamic programming using model-free Bellman Residual Elimination.” American Control Conference (ACC), 2010. 2010. 4146-4151. Print.
Version: Final published version
Other identifiers
INSPEC Accession Number: 11508712
ISBN
978-1-4244-7426-4
ISSN
0743-1619

Collections
  • MIT Open Access Articles

Browse

All of DSpaceCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

My Account

Login

Statistics

OA StatisticsStatistics by CountryStatistics by Department
MIT Libraries
PrivacyPermissionsAccessibilityContact us
MIT
Content created by the MIT Libraries, CC BY-NC unless otherwise noted. Notify us about copyright concerns.