Optimal Reissue Policies for Reducing Tail Latency
Name
reissuepolicy_preprint.pdf
Size
2.6 MB
Format
Adobe PDF
Checksum (MD5)
e583c442d2a40962e206a5432d1d26bc
Author(s)
Elnikety, Sameh
Kaler, Timothy
He, Yuxiong
Date Issued
July 2017
Journal
Proceedings of the 29th ACM Symposium on Parallelism in Algorithms and Architectures - SPAA '17
Citation
Kaler, Tim, Yuxiong He, and Sameh Elnikety. “Optimal Reissue Policies for Reducing Tail Latency.” Proceedings of the 29th ACM Symposium on Parallelism in Algorithms and Architectures - SPAA ’17 (2017).
Version
Author's final manuscript
Abstract
Interactive services send redundant requests to multiple different replicas to meet stringent tail latency requirements. These addi- tional (reissue) requests mitigate the impact of non-deterministic delays within the system and thus increase the probability of re- ceiving an on-time response. There are two existing approaches of using reissue requests to reduce tail latency. (1) Reissue requests immediately to one or more replicas, which multiplies the load and runs the risk of overloading the system. (2) Reissue requests if not completed after a fixed delay. The delay helps to bound the number of extra reissue requests, but it also reduces the chance for those requests to respond before a tail latency target. We introduce a new family of reissue policies, Single-Time / Random ( SingleR ), that reissue requests after a delay d with probability q . SingleR employs randomness to bound the reissue rate, while allowing requests to be reissued early enough so they have sufficient time to respond, exploiting the benefits of both immediate and delayed reissue of prior work. We formally prove, within a simplified analytical model, that SingleR is optimal even when compared to more complex policies that reissue multiple times. To use SingleR for interactive services, we provide efficient algorithms for calculating optimal reissue delay and probability from response time logs through data-driven approach. We apply itera- tive adaptation for systems with load-dependent queuing delays. The key advantage of this data-driven approach is its wide applica- bility and effectiveness to systems with various design choices and workload properties. We evaluated SingleR policies thoroughly. We use simulation to illustrate its internals and demonstrate its robustness to a wide range of workloads. We conduct system experiments on the Re- dis key-value store and Lucene search server. The results show that for utilizations ranging from 40 - 60% , SingleR reduces the 99 th-percentile latency of Redis by 30 - 70% by reissuing only 2% of requests, and the 99 th-percentile latency of Lucene by 15 - 25% by reissuing 1% only.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology. Department of Mathematics
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3087556.3087566
