Weld : fast data-parallel computation on modern hardware

Abstract

Modern hardware is difficult to use efficiently, requiring complex optimizations like vectorization, loop blocking and load balancing to get good performance. As a result, many widely used data processing systems fall well short of peak hardware performance. We have developed Weld, an intermediate language and runtime that can run data-parallel computations efficiently on modern hardware. The core of Weld is a novel intermediate language (IL) that is expressive enough to capture common data-parallel applications (e.g., SQL, graph analytics and machine learning) while being easy to parallelize on modern hardware, through the use of a simple "parallel builder" abstraction and nested parallel loops. Weld supports complex optimizations like vectorization and loop blocking, as well as a multicore CPU backend. Finally,Weld's runtime can to optimize across library functions used in the same program, enabling further speedups that are not possible with today's disjoint libraries. In this thesis, we describe the Weld IL and then turn to the multicore CPU backend, providing a theoretical analysis suggesting that it has low overheads and showing that microbenchmarks and real-word applications like TensorFlow have excellent multicore performance when ported to run on Weld.