Getting to the point: index sets and parallelism-preserving autodiff for pointful array programming
Name
3473593.pdf
Description
Published version
Size
477 KB
Format
Adobe PDF
Checksum (MD5)
3094be16e57b4d19da31bf2da295a628
Author(s)
Paszke, Adam
Johnson, Daniel D
Duvenaud, David
Vytiniotis, Dimitrios
Radul, Alexey
Johnson, Matthew J
Ragan-Kelley, Jonathan
Maclaurin, Dougal
Date Issued
2021
Journal
Proceedings of the ACM on Programming Languages
Publisher
Association for Computing Machinery (ACM)
Citation
Paszke, Adam, Johnson, Daniel D, Duvenaud, David, Vytiniotis, Dimitrios, Radul, Alexey et al. 2021. "Getting to the point: index sets and parallelism-preserving autodiff for pointful array programming." Proceedings of the ACM on Programming Languages, 5 (ICFP).
Version
Final published version
Abstract
We present a novel programming language design that attempts to combine the clarity and safety of high-level functional languages with the efficiency and parallelism of low-level numerical languages. We treat arrays as eagerly-memoized functions on typed index sets, allowing abstract function manipulations, such as currying, to work on arrays. In contrast to composing primitive bulk-array operations, we argue for an explicit nested indexing style that mirrors application of functions to arguments. We also introduce a fine-grained typed effects system which affords concise and automatically-parallelized in-place updates. Specifically, an associative accumulation effect allows reverse-mode automatic differentiation of in-place updates in a way that preserves parallelism. Empirically, we benchmark against the Futhark array programming language, and demonstrate that aggressive inlining and type-driven compilation allows array programs to be written in an expressive, "pointful" style with little performance penalty.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Creative Commons Attribution 4.0 International license
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3473593
