Using Straggler Replication to Reduce Latency in Large-scale Parallel Computing
Name
straggler_rep_dcc.pdf
Description
Accepted version
Size
462.08 KB
Format
Unknown
Checksum (MD5)
7d2cda40264db570f1151c142c23a10c
Author(s)
Wang, Da
Joshi, Gauri
Wornell, Gregory W
Date Issued
2015
Journal
ACM SIGMETRICS Performance Evaluation Review
Publisher
Association for Computing Machinery (ACM)
Version
Author's final manuscript
Abstract
Copyright is held by author/owner(s). In cloud computing jobs consisting of many tasks run in parallel, the tasks on the slowest machines (straggling tasks) become the bottleneck in the completion of the job. One way to combat the variability in machine response time is to add replicas of straggling tasks and wait for the earliest copy to finish. Using the theory of extreme order statistics, we analyze how task replication reduces latency, and its impact on the cost of computing resources. We also propose a heuristic algorithm to search for the best replication strategies when it is difficult to model the empirical behavior of task execution time and use the proposed analysis techniques. Evaluation of the heuristic policies on Google Trace data shows a significant latency reduction compared to the replication strategy used in MapReduce.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/2847220.2847223
