This is not the latest version of this item. The latest version can be found here.
RL-QN: A Reinforcement Learning Framework for Optimal Control of Queueing Systems
Name
2011.07401.pdf
Description
Accepted version
Size
852.84 KB
Format
Adobe PDF
Checksum (MD5)
c9cf9c2844b4035f06a0187ab3272d62
Author(s)
Liu, Bai
Xie, Qiaomin
Modiano, Eytan
Date Issued
2022
Journal
ACM Transactions on Modeling and Performance Evaluation of Computing Systems
Publisher
Association for Computing Machinery (ACM)
Citation
Liu, Bai, Xie, Qiaomin and Modiano, Eytan. 2022. "RL-QN: A Reinforcement Learning Framework for Optimal Control of Queueing Systems." ACM Transactions on Modeling and Performance Evaluation of Computing Systems, 7 (1).
Version
Author's final manuscript
Abstract
With the rapid advance of information technology, network systems have become increasingly complex and hence the underlying system dynamics are often unknown or difficult to characterize. Finding a good network control policy is of significant importance to achieve desirable network performance (e.g., high throughput or low delay). In this work, we consider using model-based reinforcement learning (RL) to learn the optimal control policy for queueing networks so that the average job delay (or equivalently the average queue backlog) is minimized. Traditional approaches in RL, however, cannot handle the unbounded state spaces of the network control problem. To overcome this difficulty, we propose a new algorithm, called RL for Queueing Networks (RL-QN), which applies model-based RL methods over a finite subset of the state space while applying a known stabilizing policy for the rest of the states. We establish that the average queue backlog under RL-QN with an appropriately constructed subset can be arbitrarily close to the optimal result. We evaluate RL-QN in dynamic server allocation, routing, and switching problems. Simulation results show that RL-QN minimizes the average queue backlog effectively.
MIT Department
Massachusetts Institute of Technology. Department of Aeronautics and Astronautics
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
10.1145/3529375
