Fine-grained I/O complexity via reductions: new lower bounds, faster algorithms, and a time hierarchy
Name
LIPIcs-ITCS-2018-34.pdf
Description
Published version
Size
652.52 KB
Format
Adobe PDF
Checksum (MD5)
bdaa37f545ef2fa988adae3f39dfa44e
Author(s)
Williams, Virginia Vassilevska
Demaine, Erik
Lincoln, Andrea
Liu, Quanquan C.
Lynch, Jayson
Date Issued
2018
Citation
Williams, Virginia Vassilevska, Demaine, Erik, Lincoln, Andrea, Liu, Quanquan C. and Lynch, Jayson. 2018. "Fine-grained I/O complexity via reductions: new lower bounds, faster algorithms, and a time hierarchy."
Version
Final published version
Abstract
© Erik D. Demaine, Andrea Lincoln, Quanquan C. Liu, Jayson Lynch, and Virginia Vassilevska Williams. This paper initiates the study of I/O algorithms (minimizing cache misses) from the perspective of fine-grained complexity (conditional polynomial lower bounds). Specifically, we aim to answer why sparse graph problems are so hard, and why the Longest Common Subsequence problem gets a savings of a factor of the size of cache times the length of a cache line, but no more. We take the reductions and techniques from complexity and fine-grained complexity and apply them to the I/O model to generate new (conditional) lower bounds as well as faster algorithms. We also prove the existence of a time hierarchy for the I/O model, which motivates the fine-grained reductions. Using fine-grained reductions, we give an algorithm for distinguishing 2 vs. 3 diameter and radius that runs in O(|E|2/(MB)) cache misses, which for sparse graphs improves over the previous O(|V |2/B) running time. We give new reductions from radius and diameter to Wiener index and median. These reductions are new in both the RAM and I/O models. We show meaningful reductions between problems that have linear-time solutions in the RAM model. The reductions use low I/O complexity (typically O(n/B)), and thus help to finely capture the relationship between “I/O linear time” (n/B) and RAM linear time (n). We generate new I/O assumptions based on the di culty of improving sparse graph problem running times in the I/O model. We create conjectures that the current best known algorithms for Single Source Shortest Paths (SSSP), diameter, and radius are optimal. From these I/O-model assumptions, we show that many of the known reductions in the word-RAM model can naturally extend to hold in the I/O model as well (e.g., a lower bound on the I/O complexity of Longest Common Subsequence that matches the best known running time). We prove an analog of the Time Hierarchy Theorem in the I/O model, further motivating the study of fine-grained algorithmic di erences.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Creative Commons Attribution 4.0 International license
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.4230/LIPIcs.ITCS.2018.34
