Optimal Network Control in Partially-Controllable Networks
Name
CV_C_221.pdf
Description
Accepted version
Size
1.16 MB
Format
Adobe PDF
Checksum (MD5)
52e3cee7f59c5979ebee9af74c9a624a
Author(s)
Liang, Qingkai
Modiano, Eytan H
Date Issued
April 2019
Journal
IEEE INFOCOM 2019
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Liang, Qingkai and Eytan Modiano. “Optimal Network Control in Partially-Controllable Networks.” Paper presented at IEEE INFOCOM 2019, Paris, France, April 29-May 2, 2019 © 2019 The Author(s)
Version
Author's final manuscript
Abstract
The effectiveness of many optimal network control algorithms (e.g., BackPressure) relies on the premise that all of the nodes are fully controllable. However, these algorithms may yield poor performance in a partially-controllable network where a subset of nodes are uncontrollable and use some unknown policy. Such a partially-controllable model is of increasing importance in real-world networked systems such as overlay-underlay networks. In this paper, we design optimal network control algorithms that can stabilize a partially-controllable network. We first study the scenario where uncontrollable nodes use a queue-agnostic policy, and propose a low-complexity throughput-optimal algorithm, called Tracking-MaxWeight (TMW), which enhances the original MaxWeight algorithm with an explicit learning of the policy used by uncontrollable nodes. Next, we investigate the scenario where uncontrollable nodes use a queue-dependent policy and the problem is formulated as an MDP with unknown queueing dynamics. We propose a new reinforcement learning algorithm, called Truncated Upper Confidence Reinforcement Learning (TUCRL), and prove that TUCRL achieves tunable three-way tradeoffs between throughput, delay and convergence rate.
MIT Department
Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
Massachusetts Institute of Technology. Department of Aeronautics and Astronautics
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/INFOCOM.2019.8737528
