An analysis of the search spaces for generate and validate patch generation systems
Author(s)
Long, Fan; Rinard, Martin C
DownloadRinard_An analysis.pdf (273.5Kb)
OPEN_ACCESS_POLICY
Open Access Policy
Creative Commons Attribution-Noncommercial-Share Alike
Terms of use
Metadata
Show full item recordAbstract
We present the first systematic analysis of key characteristics of patch search spaces for automatic patch generation systems. We analyze sixteen different configurations of the patch search spaces of SPR and Prophet, two current state-of-the-art patch generation systems. The analysis shows that 1) correct patches are sparse in the search spaces (typically at most one correct patch per search space per defect), 2) incorrect patches that nevertheless pass all of the test cases in the validation test suite are typically orders of magnitude more abundant, and 3) leveraging information other than the test suite is therefore critical for enabling the system to successfully isolate correct patches.
We also characterize a key tradeoff in the structure of the search spaces. Larger and richer search spaces that contain correct patches for more defects can actually cause systems to find fewer, not more, correct patches. We identify two reasons for this phenomenon: 1) increased validation times because of the presence of more candidate patches and 2) more incorrect patches that pass the test suite and block the discovery of correct patches. These fundamental properties, which are all characterized for the first time in this paper, help explain why past systems often fail to generate correct patches and help identify challenges, opportunities, and productive future directions for the field.
Date issued
2016-05Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer ScienceJournal
Proceedings of the 38th International Conference on Software Engineering - ICSE '16
Publisher
Association for Computing Machinery
Citation
Long, Fan, and Martin Rinard. "An Analysis of the Search Spaces for Generate and Validate Patch Generation Systems." Proceedings of the 38th International Conference on Software Engineering - ICSE '16, 14-22 May, 2016, Austin, Texas, ACM Press, 2016, pp. 702–13.
Version: Author's final manuscript
ISSN
978-1-4503-3900-1