Chasing zero variability in software performance

Abstract

The ability to understand and control software performance variability is important for writing programs that reliably meet performance requirements. It is also crucial for effective performance engineering because it allows the programmer to collect fewer datapoints and still draw statistically significant conclusions. This wastes less developer time and fewer resources, and additionally makes processes like autotuning significantly more practical. Unfortunately, performance variability is often seen as unavoidable fact of commodity computing systems. This thesis challenges that notion, and shows that we can obtain 0-cycle variability for CPU-bound workloads, and <0.3% variability for workloads that touch memory. It shows how a programmer might take an arbitrary system and tease out and address sources of variability, and also provides a comprehensive glossary of common causes, making it a useful guide for the practical performance engineer.