Automated feedback generation for introductory programming assignments
Author(s)Singh, Rishabh; Gulwani, Sumit; Solar-Lezama, Armando
MetadataShow full item record
We present a new method for automatically providing feedback for introductory programming problems. In order to use this method, we need a reference implementation of the assignment, and an error model consisting of potential corrections to errors that students might make. Using this information, the system automatically derives minimal corrections to student's incorrect solutions, providing them with a measure of exactly how incorrect a given solution was, as well as feedback about what they did wrong. We introduce a simple language for describing error models in terms of correction rules, and formally define a rule-directed translation strategy that reduces the problem of finding minimal corrections in an incorrect program to the problem of synthesizing a correct program from a sketch. We have evaluated our system on thousands of real student attempts obtained from the Introduction to Programming course at MIT (6.00) and MITx (6.00x). Our results show that relatively simple error models can correct on average 64% of all incorrect submissions in our benchmark set.
DepartmentMassachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation -- PLDI '13
Association for Computing Machinery
Singh, Rishabh, Sumit Gulwani, and Armando Solar-Lezama. “Automated Feedback Generation for Introductory Programming Assignments.” 34th ACM SIGPLAN conference on Programming language design and implementation, PLDI'13, June 16-19, 2013, Seattle, WA, USA. p.15-26.
Author's final manuscript