Bug-Injecting System Helps to Advance the State-of-the-Art in Debugging Software

Abstract

Bug finding systems are used after developers have written code to try to identify mistakes they have made. If these systems find a bug, they can be fixed before code is deployed. Unfortunately, these systems fail to find many bugs, which is one of the reasons why new vulnerabilities and crashes still exist in computer programs today. The scarce documentation of known bugs and how those bugs manifest in a program made it impossible to measure the success of bug-finding tools.

The Large-scale Automated Vulnerability Addition (LAVA) system enables evaluation of bug-finding systems. The LAVA system produces thousands of realistic bugs that are automatically injected into pre-existing program code. Once these bugs are injected, various vulnerability discovery techniques and software can be tested to see how many of the bugs are found and how many are missed.

Over the last five years, LAVA has become the first widely used benchmark for evaluation of bug-finding systems.