Designing an experimental platform for a split-classroom comparison study

Abstract

SimBio, an education-technology company, has created two new online question formats which are constrained enough to give students automatic feedback, but open enough to give students ample room for expression. The LabLib, modeled after a MadLib, consists of multiple fill-in-the-blank answer choices, which students select from a drop-down. The WordBytes question format, modeled after refrigerator poetry, consists of pre-determined tiles that students can drag and drop to form their answer. Automatic feedback is provided on-demand in the form of a popup window. This thesis presents the design and implementation of the first-ever split-classroom study to compare the LabLib and WordBytes format to each other. An experimental platform was created to support the study's requirements, and a 319-participant study was conducted with high school students from the Greater Boston area. We evaluated the LabLibs and WordBytes on learning gains, completion rates, and student engagement. Preliminary data reveals no significant difference in learning gains between the two formats. However, completion rates of questions in the LabLib and WordBytes format were higher than that of questions in a Free Response format. The data also suggests students were engaged with the LabLib and WordBytes questions, measured by the number of times students checked their answers.