Risk-sensitive optimal control of queues

Abstract

We consider the problem of designing risk-sensitive optimal control policies for scheduling packet transmissions in a stochastic wireless network. A single client is connected to an access point (AP) through a wireless channel. Packet transmission incurs a cost C, while packet delivery yields a reward of R units. The client maintains a finite buffer of size B, and a penalty of L units is imposed upon packet loss which occurs due to buffer overflow. We show that the risk-sensitive optimal control policy for such a simple set-up is of threshold type, i.e., it is optimal to carry out packet transmissions only when Q(t), i.e., the queue length at time t exceeds a certain threshold t. It is also shown that the value of the threshold t increases upon increasing the cost per unit packet transmission C. Furthermore, it is also shown that a threshold policy with threshold equal to t is optimal for a set of problems in which cost C lies within an interval [Cl, Cu]. Equations that need to be solved in order to obtain C¡, Cu are also provided.