An analysis of the search spaces for generate and validate patch generation systems
Name
Rinard_An analysis.pdf
Size
273.6 KB
Format
Adobe PDF
Checksum (MD5)
48b232fc27a63d30133d8dd353cd4bf1
Author(s)
Long, Fan
Rinard, Martin C
Date Issued
May 2016
Journal
Proceedings of the 38th International Conference on Software Engineering - ICSE '16
Publisher
Association for Computing Machinery
Citation
Long, Fan, and Martin Rinard. "An Analysis of the Search Spaces for Generate and Validate Patch Generation Systems." Proceedings of the 38th International Conference on Software Engineering - ICSE '16, 14-22 May, 2016, Austin, Texas, ACM Press, 2016, pp. 702–13.
Version
Author's final manuscript
Abstract
We present the first systematic analysis of key characteristics of patch search spaces for automatic patch generation systems. We analyze sixteen different configurations of the patch search spaces of SPR and Prophet, two current state-of-the-art patch generation systems. The analysis shows that 1) correct patches are sparse in the search spaces (typically at most one correct patch per search space per defect), 2) incorrect patches that nevertheless pass all of the test cases in the validation test suite are typically orders of magnitude more abundant, and 3) leveraging information other than the test suite is therefore critical for enabling the system to successfully isolate correct patches.
We also characterize a key tradeoff in the structure of the search spaces. Larger and richer search spaces that contain correct patches for more defects can actually cause systems to find fewer, not more, correct patches. We identify two reasons for this phenomenon: 1) increased validation times because of the presence of more candidate patches and 2) more incorrect patches that pass the test suite and block the discovery of correct patches. These fundamental properties, which are all characterized for the first time in this paper, help explain why past systems often fail to generate correct patches and help identify challenges, opportunities, and productive future directions for the field.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/2884781.2884872
