Modeling Human Planning in Maze Orienteering Problems

Abstract

To create socially intelligent artificial assistants for humans in complex, naturalistic search environments, we need to develop algorithms that build models of human planning given their past decisions. In this thesis project, I focused on modeling human planning in Maze Orienteering Problems (MOP), an optimization problem with the objective to maximize collected rewards within a time limit in a partially known maze.

The project has two main components: developing planning algorithms to find approximate solutions to the MOP and using those algorithms to model human behavior with Bayesian inference.

For the planning part, I designed a hierarchical planning framework to solve the MOP as a room-level orienteering problem and a Partially Observable Markov Decision Process (POMDP) inside each room. My evaluation of algorithms shows that a Closest-Room heuristic model for room-level planning performs comparable to Branch-and-Bound exhaustive search while bearing a much smaller computational cost.

For the inference part, I implemented an online Bayesian inverse planning framework to fit candidate hierarchical planners to individual human traces. My experiments of human modeling shows that Closest-Room heuristic model also outperforms BnB in fitting humans’ room-level decisions and predicting their next rooms to visit.