Modeling problem solving in Massive Open Online Courses

Abstract

.Massive Open Online Courses (MOOC) have presented a completely new style of learning and teaching that also brings us a massive amount of student behavioral data. Some of this data is exclusive to the MOOC environment. It opens up many possibilities for educators to study a question they have always wanted to answer: how do students solve problems? In this thesis, we present and address some of the numerous challenges one encounters during the process of mining MOOC data to answer this seemingly simple question. We describe in detail, using the data from MITx's 6.002x Spring 2012 course offering, a large scale, mixed automated and manual process that starts with the re-organization of MOOCdb source data into relevant and retrieval-efficient abstractions we call student resource trajectories and answer type transition matrices. This step must be interleaved with meticulous and painstaking automatic and manual curation of the data to remove errors and irrelevancies while aggregating redundancies, reducing noise and assuring meaningful, trustworthy variables. Regardless, only an estimation of student resource usage behavior during problem solving is available. With all student trajectories for every problem of 6.002X extracted, we demonstrate some analyses of student behaviors for the whole student population. These offer some insight into a problem's level of difficulty and student behavior around a problem type, such as homework. Next, in order to study how students reached the correct solution to a problem, we categorize problem answers and consider how student move from one incorrect answer to their next attempt. This requires extensive filtering out of irrelevancies and rankings. Detailed knowledge of resources, as would be expected of an instructor, appears to be crucial to understanding the implications of the statistics we derive on frequency of resource usage in general and per attempt. We identify solution equivalence and interpretation also as significant hurdles in obtaining insights. Finally, we try to describe students' problem solving process in terms of resource use patterns by using hidden Markov modeling with original variable definitions and 3 different variable relationships (graphical structures). We evaluate how well these models actually describe the student trajectories and try to use them to predict upcoming student submission events on 24 different selected homework problems. The model with the most complex variable relationships proves to be most accurate.