Securing software : an evaluation of static source code analyzers

Abstract

This thesis evaluated five static analysis tools--Polyspace C Verifier, ARCHER, BOON, Splint, and UNO--using 14 code examples that illustrated actual buffer overflow vulnerabilities found in various versions of Sendmail, BIND, and WU-FTPD. Each code example included a "BAD" case with one or more buffer overflow vulnerabilities and a "PATCHED" case without buffer overflows. The buffer overflows varied and included stack, heap, bss and data buffers; access above and below buffer bounds; access using pointers, indices, and functions; and scope differences between buffer creation and use. Detection rates for the "BAD" examples were low except for Splint and PolySpace C Verifier, which had average detection rates of 57% and 87% respectively. However, average false alarm rates, as measured using the "PATCHED" programs, were high for these two systems. The frequency of false alarms per lines of code was high for both of these tools; Splint gave on average one false alarm per 50 lines of code, and PolySpace gave on average one false alarm per 10 lines of code. This result shows that current approaches can detect buffer overflows, but that false alarm rates need to be lowered substantially.