Repository logo
Log in(current)
Repository logoMIT Open ScholarshipDSpace@MIT
  1. Home
  2. MIT Open Access Articles
  3. MIT Open Access Articles
  4. Scheduling Distributed Clusters of Parallel Machines : Primal-Dual and LP-based Approximation Algorithms

Scheduling Distributed Clusters of Parallel Machines : Primal-Dual and LP-based Approximation Algorithms

Thumbnail Image
Download
Name

453_2017_345_ReferencePDF.pdf

Size

679.64 KB

Format

Adobe PDF

Checksum (MD5)

ebb924ee2dab4896f151eec1b94b7842

Download all files submitted through automated deposit
art_897953424242447235.zip (627.59 KB)
Author(s)
Murray, Riley
•
Khuller, Samir
•
Chao, Megan C.
Date Issued
July 2017
Journal
Algorithmica
Publisher
Springer-Verlag
Citation
Murray, Riley, Samir Khuller, and Megan Chao. “Scheduling Distributed Clusters of Parallel Machines : Primal-Dual and LP-Based Approximation Algorithms.” Algorithmica 80, no. 10 (July 19, 2017): 2777–2798.
Version
Author's final manuscript
Abstract
The Map-Reduce computing framework rose to prominence with datasets of such size that dozens of machines on a single cluster were needed for individual jobs. As datasets approach the exabyte scale, a single job may need distributed processing not only on multiple machines, but on multiple clusters. We consider a scheduling problem to minimize weighted average completion time of n jobs on m distributed clusters of parallel machines. In keeping with the scale of the problems motivating this work, we assume that (1) each job is divided into m “subjobs” and (2) distinct subjobs of a given job may be processed concurrently. When each cluster is a single machine, this is the NP-Hard concurrent open shop problem. A clear limitation of such a model is that a serial processing assumption sidesteps the issue of how different tasks of a given subjob might be processed in parallel. Our algorithms explicitly model clusters as pools of resources and effectively overcome this issue. Under a variety of parameter settings, we develop two constant factor approximation algorithms for this problem. The first algorithm uses an LP relaxation tailored to this problem from prior work. This LP-based algorithm provides strong performance guarantees. Our second algorithm exploits a surprisingly simple mapping to the special case of one machine per cluster. This mapping-based algorithm is combinatorial and extremely fast. These are the first constant factor approximations for this problem.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
http://hdl.handle.net/1721.1/116477
DOI of Published Version
https://doi.org/10.1007/s00453-017-0345-x
Repository logo
PrivacyPermissionsAccessibilityContact us
Repository logo
Notify us about copyright concerns.