Languages and Compilers for Rendering and Image Processing

Abstract

Even though computer graphics applications are widely used, they remain challenging to implement and graphics programming systems must navigate conflicting trade-offs between correctness, performance, and hardware portability. This thesis describes the design and implementation of domain specific languages with particular trade-off decisions in mind and the application of machine learning to these languages and their compilers.

Rendering systems suffer from a tension between separation of concerns and performance. Existing rendering systems typically focus on performance, but complex probability computations make advanced rendering algorithms difficult to implement correctly. We first identify some common operations that are foundational to many rendering algorithms, describe some goals of an ideal rendering system, explore the space of trade-offs in achieving these goals, and discuss some possible implementation strategies. We then present Aether, a domain specific language for rendering, designed with a focus on correctness. Users write sampling code using reusable building block components and all probability code is then automatically generated. We demonstrate the effectiveness of this approach by implementing a range of modern rendering algorithms, including the novel tridirectional path tracing, which otherwise would have been prohibitively difficult to implement.

Halide provides a modular approach to writing image processing code but achieving high performance still requires considerable manual effort and expertise. We present a new automatic algorithm that quickly generates high performance GPU implementations of imaging and vision pipelines, directly from high-level Halide algorithm code. We address the scalability challenge of extending search-based automatic scheduling to the nested parallelism on GPU architectures in reasonable compile time. We find schedules that are on average 1.7x faster than existing automatic solutions (up to 5x), and competitive with what the best human experts were able to achieve in an active effort to beat our automatic results.