Point-Based Policy Transformation: Adapting Policy to Changing POMDP Models
Name
wafr12_polDeform.pdf
Size
412.04 KB
Format
Adobe PDF
Checksum (MD5)
ebd7fbdb3ee4cd6dcaa6a0d39c7d4826
Author(s)
Kurniawati, Hanna
Patrikalakis, Nicholas M
Date Issued
2013
Journal
Algorithmic Foundations of Robotics X
Publisher
Springer Nature America, Inc
Citation
Kurniawati, Hanna, and Nicholas M. Patrikalakis. “Point-Based Policy Transformation: Adapting Policy to Changing POMDP Models.” Algorithmic Foundations of Robotics X (2013): 493–509.
Version
Author's final manuscript
Abstract
Motion planning under uncertainty that can efficiently take into account changes in the environment is critical for robots to operate reliably in our living spaces. Partially Observable Markov Decision Process (POMDP) provides a systematic and general framework for motion planning under uncertainty. Point-based POMDP has advanced POMDP planning tremendously over the past few years, enabling POMDP planning to be practical for many simple to moderately difficult robotics problems. However, when environmental changes alter the POMDP model, most existing POMDP planners recompute the solution from scratch, often wasting significant computational resources that have been spent for solving the original problem. In this paper, we propose a novel algorithm, called Point-Based Policy Transformation (PBPT), that solves the altered POMDP problem by transforming the solution of the original problem to accommodate changes in the problem. PBPT uses the point-based POMDP approach. It transforms the original solution by modifying the set of sampled beliefs that represents the belief space B, and then uses this new set of sampled beliefs to revise the original solution. Preliminary results indicate that PBPT generates a good policy for the altered POMDP model in a matter of minutes, while recomputing the policy using the fastest offline POMDP planner today fails to find a policy with similar quality after two hours of planning time, even when the policy for the original problem is reused as an initial policy. Keywords: Optimal Policy, Autonomous Underwater Vehicle, Reward Function, Good Policy State Trace
MIT Department
Massachusetts Institute of Technology. Department of Mechanical Engineering
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1007/978-3-642-36279-8_30
