On maximum-reward motion in stochastic environments

Abstract

In this thesis, we consider the problem of an autonomous mobile robot operating in a stochastic reward field to maximize total rewards collected in an online setting. This is a generalization of the problem where an unmanned aerial vehicle (UAV) collects data from randomly deployed unattended ground sensors (UGS). Specifically, the rewards are assumed to be generated by a Poisson point process. The robot has a limited perception range, and thus it discovers the reward field on the fly. The robot is assumed to be a dynamical system with substantial drift in one direction, e.g., a high-speed airplane, so it cannot traverse the entire field. The task of the robot is to maximize the total rewards collected during the course of the mission, given above constraints. Under such assumptions, we analyze the performance of a simple receding-horizon planning algorithm with respect to the perception range, robot agility and computational resources available. Firstly, we show that, with highly limited perception range, the robot is able to collect as many rewards as if it could see the entire reward field, if and only if the reward distribution is light-tailed. The second result attained shows that the expected rewards collected scale proportionally to the square root of the robot agility. Finally, we are able to prove that the overall computational workload increases linearly with the mission length, i.e., the distance of travel. We verify our results in simulation examples. At the end, we present one interesting application of our theoretical study to the ground sensor selection problem. For an inference/estimation task, we prove that sensors with randomized quality outperform those with homogeneous precisions, since random sensors yield a higher confidence level of estimation (lower variance), under certain technical assumptions. This finding might have practical implications on the design of UAV-UGS systems.