Flexible Queueing Architectures

Abstract

We study a multiserver model with n flexible servers and n queues, connected through a bipartite graph, where the level of flexibility is captured by an upper bound on the graph's average degree, dn. Applications in content replication in data centers, skillbased routing in call centers, and flexible supply chains are among our main motivations. We focus on the scaling regime where the system size n tends to infinity, while the overall traffic intensity stays fixed. We show that a large capacity region and an asymptotically vanishing queueing delay are simultaneously achievable even under limited flexibility (d n ≪n). Our main results demonstrate that, when d n ≫ln n, a family of expander-graphbased flexibility architectures has a capacity region that is within a constant factor of the maximum possible, while simultaneously ensuring a diminishing queueing delay for all arrival rate vectors in the capacity region. Our analysis is centered around a new class of virtual-queue-based scheduling policies that rely on dynamically constructed job-toserver assignments on the connectivity graph. For comparison, we also analyze a natural family of modular architectures, which is simpler but has provably weaker performance.