PipeRAG: Fast Retrieval-Augmented Generation via Adaptive Pipeline Parallelism
Name
3690624.3709194.pdf
Size
3.17 MB
Format
Adobe PDF
Checksum (MD5)
1c9fc6fa361de5dc62d610d63deac63d
Author(s)
Jiang, Wenqi
Zhang, Shuai
Han, Boran
Wang, Jie
Wang, Bernie
Kraska, Tim
Date Issued
July 20, 2025
Publisher
ACM|Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1
Citation
Wenqi Jiang, Shuai Zhang, Boran Han, Jie Wang, Bernie Wang, and Tim Kraska. 2025. PipeRAG: Fast Retrieval-Augmented Generation via Adaptive Pipeline Parallelism. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1 (KDD '25). Association for Computing Machinery, New York, NY, USA, 589–600.
Version
Final published version
Abstract
Retrieval-augmented generation (RAG) can enhance the generation quality of large language models (LLMs) by incorporating external token databases. However, retrievals from large databases can constitute a substantial portion of the overall generation time, particularly when retrievals are periodically performed to align the retrieved content with the latest states of generation. In this paper, we introduce PipeRAG, a novel algorithm-system co-design approach to reduce generation latency and enhance generation quality. PipeRAG integrates (1) pipeline parallelism to enable concurrent retrieval and generation processes, (2) flexible retrieval intervals to maximize the efficiency of pipeline parallelism, and (3) a performance model to automatically balance retrieval quality and latency based on the generation states and underlying hardware. Our evaluation shows that, by combining the three aforementioned methods, PipeRAG achieves up to 2.6× speedup in end-to-end generation latency while improving generation quality. These promising results showcase the effectiveness of co-designing algorithms with underlying systems, paving the way for the adoption of PipeRAG in future RAG systems.
Description
KDD ’25, August 3–7, 2025, Toronto, ON, Canada
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3690624.3709194
