An order-aware dataflow model for parallel Unix pipelines
Name
3473570.pdf
Description
Published version
Size
702.95 KB
Format
Adobe PDF
Checksum (MD5)
2ae8518c4884558d6a82d07abca4f07d
Author(s)
Handa, Shivam
Kallas, Konstantinos
Vasilakis, Nikos
Rinard, Martin C
Date Issued
2021
Journal
Proceedings of the ACM on Programming Languages
Publisher
Association for Computing Machinery (ACM)
Citation
Handa, Shivam, Kallas, Konstantinos, Vasilakis, Nikos and Rinard, Martin C. 2021. "An order-aware dataflow model for parallel Unix pipelines." Proceedings of the ACM on Programming Languages, 5 (ICFP).
Version
Final published version
Abstract
We present a dataflow model for modelling parallel Unix shell pipelines. To accurately capture the semantics of complex Unix pipelines, the dataflow model is order-aware, i.e., the order in which a node in the dataflow graph consumes inputs from different edges plays a central role in the semantics of the computation and therefore in the resulting parallelization. We use this model to capture the semantics of transformations that exploit data parallelism available in Unix shell computations and prove their correctness. We additionally formalize the translations from the Unix shell to the dataflow model and from the dataflow model back to a parallel shell script. We implement our model and transformations as the compiler and optimization passes of a system parallelizing shell pipelines, and use it to evaluate the speedup achieved on 47 pipelines.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Creative Commons Attribution 4.0 International license
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3473570
