Optimization of naïve dynamic binary instrumentation Tools/

Abstract

The proliferation of dynamic program analysis tools has done much to ease the burden of developing complex software. However, creating such tools remains a challenge. Dynamic binary instrumentation frameworks such as DyanamoRIO and Pin provide support for such tools by taking responsibility for application transparency and machine code manipulation. However, tool writers must still make a tough choice when writing instrumentation: should they inject custom inline assembly into the application code, or should they use the framework facilities for inserting callbacks into regular C code? Custom assembly can be more performant and more flexible, but it forces the tool to take some responsibility for maintaining application transparency. Callbacks into C, or "clean calls," allow the tool writer to ignore the details of maintaining transparency. Generally speaking, a clean call entails switching to a safe stack, saving all registers, materializing the arguments, and jumping to the callback. This thesis presents a suite of optimizations for DynamoRIO that improves the performance of "naive tools," or tools which rely primarily on clean calls for instrumentation. Most importantly, we present a novel partial inlining optimization for instrumentation routines with conditional analysis. For simpler instrumentation routines, we present a novel call coalescing optimization that batches calls into fewer context switches. In addition to these two novel techniques, we provide a suite of machine code optimizations designed to leverage the opportunities created by the aforementioned techniques. With this additional functionality built on DynamoRIO, we have shown improvements of up to 54.8x for a naive instruction counting tool as well as a 3.7x performance improvement for a memory alignment checking tool on average for many of the benchmarks from the SPEC 2006 CPU benchmark suite.