Fuzzing for User-Schedulable Languages

Abstract

Performance engineers restructure programs to use hardware as efficiently as possible. Even simple mathematical functions can become sprawling and complex programs when fully optimized, as the resulting code must often be precisely molded around specialized behaviors supported by the hardware. To help performance engineers deal with this complexity, userschedulable languages provide scheduling operations, which are abstractions of common steps taken to restructure programs. By composing these scheduling operations, performance engineers can concisely represent their intended optimizations to programs. Exo, being a user-schedulable language, provides this abstraction with the additional guarantee that any scheduling operation which passes Exo’s automated checks does not change the behavior of the program. Though this guarantee is useful for avoiding bugs while optimizing a program, the analysis required to provide such a guarantee is infeasible on programs in general. To make analysis feasible, Exo only allows users to write programs with a restricted set of behaviors. As a result, some programs are impossible to schedule using Exo, limiting the use cases of Exo. In this thesis, we explore how fuzzing can be used as an alternative to the existing analysis in Exo, with the goal of allowing Exo to analyze more complex programs. “Fuzzing” refers to a test case-driven approach to determining properties of a program, such as whether its behavior changes after a scheduling operation. If the program’s outputs do not change after the scheduling operation when provided the same inputs, the fuzzer concludes that the program’s behavior did not change. Since fuzzing only requires us to know how to evaluate the program, it can be applied to a much broader set of programs than the existing analysis in Exo. However, fuzzing can miss mistakes in scheduling if the fuzzer fails to find a test case demonstrating the issue with a scheduling operation, as it is a complete form of analysis rather than a sound form of analysis like the existing analysis in Exo. Additionally, fuzzing can be costly compared to the original analysis, as repeatedly running programs on many test cases for many scheduling operations can be slow. We explore ways to mitigate these issues throughout this work. Finally, we evaluate our implementation of the fuzzer and its performance on some example use cases for Exo.