MIT Libraries logoDSpace@MIT

MIT
View Item 
  • DSpace@MIT Home
  • Computer Science and Artificial Intelligence Lab (CSAIL)
  • Artificial Intelligence Lab Publications
  • AI Technical Reports (1964 - 2004)
  • View Item
  • DSpace@MIT Home
  • Computer Science and Artificial Intelligence Lab (CSAIL)
  • Artificial Intelligence Lab Publications
  • AI Technical Reports (1964 - 2004)
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

Importance Sampling for Reinforcement Learning with Multiple Objectives

Author(s)
Shelton, Christian Robert
Thumbnail
DownloadAITR-2001-003.ps (10.06Mb)
Additional downloads
AITR-2001-003.pdf (1.209Mb)
Metadata
Show full item record
Abstract
This thesis considers three complications that arise from applying reinforcement learning to a real-world application. In the process of using reinforcement learning to build an adaptive electronic market-maker, we find the sparsity of data, the partial observability of the domain, and the multiple objectives of the agent to cause serious problems for existing reinforcement learning algorithms. We employ importance sampling (likelihood ratios) to achieve good performance in partially observable Markov decision processes with few data. Our importance sampling estimator requires no knowledge about the environment and places few restrictions on the method of collecting data. It can be used efficiently with reactive controllers, finite-state controllers, or policies with function approximation. We present theoretical analyses of the estimator and incorporate it into a reinforcement learning algorithm. Additionally, this method provides a complete return surface which can be used to balance multiple objectives dynamically. We demonstrate the need for multiple goals in a variety of applications and natural solutions based on our sampling method. The thesis concludes with example results from employing our algorithm to the domain of automated electronic market-making.
Date issued
2001-08-01
URI
http://hdl.handle.net/1721.1/5568
Other identifiers
AITR-2001-003
CBCL-204
Series/Report no.
AITR-2001-003CBCL-204
Keywords
AI, reinforcement learning, RL, importance sampling, estimation, market-making

Collections
  • AI Technical Reports (1964 - 2004)
  • CBCL Memos (1993 - 2004)

Browse

All of DSpaceCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

My Account

Login

Statistics

OA StatisticsStatistics by CountryStatistics by Department
MIT Libraries
PrivacyPermissionsAccessibilityContact us
MIT
Content created by the MIT Libraries, CC BY-NC unless otherwise noted. Notify us about copyright concerns.