HyProf: A Profiler for Programming Students that Offers Hypotheses about Performance Bugs

Abstract

Programming students often struggle to find and fix performance bugs in their code. To provide students additional performance debugging support, as well as expose them to profiling tools, we developed Hypothesis Profiler (HyProf). HyProf automatically profiles a slow student submission and produces a profile visualization suitable for learners. In addition to showing individual function and line times, HyProf shows details about the call graph, lines that made recursive calls or did not execute, and hypotheses about possible causes of slow performance, formulated by comparing the slow profile against fast submissions from other students. We deployed HyProf in a 400-student Python course and evaluated it through web logs, office hour observations, and surveys, which showed that 75% of respondents successfully used HyProf to find or fix a performance issue and 85% would recommend it to others.