Point-Based Policy Transformation: Adapting Policy to Changing POMDP Models
Author(s)
Kurniawati, Hanna; Patrikalakis, Nicholas M
Downloadwafr12_polDeform.pdf (412.0Kb)
OPEN_ACCESS_POLICY
Open Access Policy
Creative Commons Attribution-Noncommercial-Share Alike
Terms of use
Metadata
Show full item recordAbstract
Motion planning under uncertainty that can efficiently take into account changes in the environment is critical for robots to operate reliably in our living spaces. Partially Observable Markov Decision Process (POMDP) provides a systematic and general framework for motion planning under uncertainty. Point-based POMDP has advanced POMDP planning tremendously over the past few years, enabling POMDP planning to be practical for many simple to moderately difficult robotics problems. However, when environmental changes alter the POMDP model, most existing POMDP planners recompute the solution from scratch, often wasting significant computational resources that have been spent for solving the original problem. In this paper, we propose a novel algorithm, called Point-Based Policy Transformation (PBPT), that solves the altered POMDP problem by transforming the solution of the original problem to accommodate changes in the problem. PBPT uses the point-based POMDP approach. It transforms the original solution by modifying the set of sampled beliefs that represents the belief space B, and then uses this new set of sampled beliefs to revise the original solution. Preliminary results indicate that PBPT generates a good policy for the altered POMDP model in a matter of minutes, while recomputing the policy using the fastest offline POMDP planner today fails to find a policy with similar quality after two hours of planning time, even when the policy for the original problem is reused as an initial policy. Keywords: Optimal Policy, Autonomous Underwater Vehicle, Reward Function, Good Policy State Trace
Date issued
2013Department
Massachusetts Institute of Technology. Department of Mechanical EngineeringJournal
Algorithmic Foundations of Robotics X
Publisher
Springer Nature America, Inc
Citation
Kurniawati, Hanna, and Nicholas M. Patrikalakis. “Point-Based Policy Transformation: Adapting Policy to Changing POMDP Models.” Algorithmic Foundations of Robotics X (2013): 493–509.
Version: Author's final manuscript
ISBN
978-3-642-36278-1
978-3-642-36279-8
ISSN
1610-7438
1610-742X