Multi-stage resource-aware scheduling for data centers with heterogeneous servers
Name
10951_2017_537_ReferencePDF.pdf
Size
753.59 KB
Format
Adobe PDF
Checksum (MD5)
7c849356dbeeb397b2bd527f0ea90256
Author(s)
Padmanabhan, Meghana
Li, Heyse
Tran, Tony T.
Down, Douglas G.
Beck, J. Christopher
Zhang, Yun
Date Issued
July 2017
Journal
Journal of Scheduling
Publisher
Springer US
Citation
Tran, Tony T., et al. “Multi-Stage Resource-Aware Scheduling for Data Centers with Heterogeneous Servers.” Journal of Scheduling, vol. 21, no. 2, Apr. 2018, pp. 251–67.
Version
Author's final manuscript
Abstract
This paper presents a three-stage algorithm for resource-aware scheduling of computational jobs in a large-scale heterogeneous data center. The algorithm aims to allocate job classes to machine configurations to attain an efficient mapping between job resource request profiles and machine resource capacity profiles. The first stage uses a queueing model that treats the system in an aggregated manner with pooled machines and jobs represented as a fluid flow. The latter two stages use combinatorial optimization techniques to solve a shorter-term, more accurate representation of the problem using the first-stage, long-term solution for heuristic guidance. In the second stage, jobs and machines are discretized. A linear programming model is used to obtain a solution to the discrete problem that maximizes the system capacity given a restriction on the job class and machine configuration pairings based on the solution of the first stage. The final stage is a scheduling policy that uses the solution from the second stage to guide the dispatching of arriving jobs to machines. We present experimental results of our algorithm on both Google workload trace data and generated data and show that it outperforms existing schedulers. These results illustrate the importance of considering heterogeneity of both job and machine configuration profiles in making effective scheduling decisions. Keywords: Resource-aware scheduling, Dynamic scheduling, Heterogeneous servers
MIT Department
Massachusetts Institute of Technology. Institute for Data, Systems, and Society
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1007/s10951-017-0537-x
