Programming technologies for engineering quality multicore software

Abstract

The widespread availability of large multicore computers in the cloud has given engineers and scientists unprecedented access to large computing platforms. Traditionally, high-end computing solutions have been developed and used by only a small community, as these solutions rely on expensive and specialized computing environments. The emergence of large-scale cloud computing providers, however, has democratized access to large-scale (although not necessarily HPC-scale) computing power, which can now be rented on-demand with just a credit card. The complexity of parallel programming, however, has made it more difficult for even expert programmers to develop high-quality multicore software systems. For average programmers, developing parallel programs that are debuggable, correct, and performant is a daunting challenge.

This thesis is concerned with the development of programming technologies that reduce the complexity of parallel programming to make it easier for average programmers to exploit the capabilities of multicore hardware. I contend that realizing the full potential of the multicore revolution requires the development of programming technologies that make it easier to write quality code -- code that has a simple understandable structure and performs well in practice. These programming technologies broadly include parallel algorithms, data structures, optimization techniques, profiling tools, and system design principles. Along these ends, this thesis presents seven intellectual artifacts from the domains of parallel algorithms, multicore-centric systems for scientific computing, and programming tools that make it easier to write quality code by simplifying the design, analysis, and performance engineering of multicore software: --

Chromatic: Parallel algorithms for scheduling data-graph computations deterministically. -- Color: Parallel algorithms and ordering heuristics for graph coloring that have the simple semantics of serial code. -- PARAD: An ecient and parallelism-preserving algorithm for performing automatic differentiation in parallel programs. -- Connectomics: An end-to-end image-segmentation pipeline for connectomics using a single large multicore. -- Alignment: An image-alignment pipeline for connectomics that uses memory-ecient algorithms, and techniques for judiciously exploiting performance{accuracy tradeoffs. -- Reissue: Reissue policies for reducing tail-latency in distributed services that are easy to analyze and eective in practice. --

Cilkmem: Ecient algorithms and tools for measuring the worst-case memory high-water mark of parallel programs. Although the emphasis and domains of these artifacts vary, they each involve the discovery of a way to tame complexity in parallel software systems without compromising, in fact, usually enhancing, theoretical guarantees and real-world performance.