CS2 Student Programming Performance Prediction and Intervention

Abstract

As the number of students in a course grows, it becomes increasingly difficult for instructors to identify and help students who are struggling to develop good understanding of the material. This study investigates scalable prediction and intervention methods in the context of 6.101, an intermediate programming course at MIT. First, a broad investigation was conducted into early predictive factors of students who earn a C, D, F, or Withdraw (CDFW) from 6.101. Results suggested that limited prior programming experience was associated with higher CDFW rates, as were other factors such as high amounts of early office hour usage and lower grades in certain prerequisites. Prediction efforts focused on students of interest (SOI) who initially committed to the course but were likely to earn a C, D, F or Later Withdraw (CDFLW) from 6.101. A hand-tuned model that combined various predictive factors identified SOI with 75 percent accuracy (13 percent sensitivity, 90 percent specificity) three weeks into the semester. In order to help SOI develop their programming skills, encourage independent problem solving, and increase feelings of belonging and community within the CS department, a series of optional weekly programming practice sessions were developed and implemented. While the results of the intervention are inconclusive due to the small number of students who attended sessions and responded to post-semester surveys, the available data from two semesters suggests that the intervention had limited impact in all three design areas. Overall SOI had lower exam scores, received more help with assignments, and reported lower ratings of belonging and community at the end of the semester compared to non-SOI. These findings have potential broader implications for how “at-risk” students are defined, how predictive models are created and used, and how interventions are designed.