Dynamic overload balancing in server farms

Abstract

We consider the problem of optimal load balancing in a server farm under overload conditions. A convex penalty minimization problem is studied to optimize queue overflow rates at the servers. We introduce a new class of α-fair penalty functions, and show that the cases of α = 0, 1, ∞ correspond to minimum sum penalty, penalty proportional fairness, and min-max fairness, respectively. These functions are useful to maximize the time to first buffer overflow and minimize the recovery time from temporary overload. In addition, we show that any policy that solves an overload minimization problem with strictly increasing penalty functions must be throughput optimal. A dynamic control policy is developed to solve the overload minimization problem in a stochastic setting. This policy generalizes the well-known join-the-shortest-queue (JSQ) policy and uses intelligent job tagging to optimize queue overflow rates without the knowledge of traffic arrival rates.