2026-09-20T16:59:30Zhttps://dspace.mit.edu/server/oai/requestoai:dspace.mit.edu:1721.1/1452242025-10-30T15:50:03Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023
Chris Caplice and Eva Ponce.
Bilal, Ahmed.
Massachusetts Institute of Technology. Engineering Systems Division.
System Design and Management Program.
Massachusetts Institute of Technology. Engineering Systems Division
System Design and Management Program.
2022-08-31T16:29:14Z
2022-08-31T16:29:14Z
2019
2019
https://hdl.handle.net/1721.1/145224
1341989968
Thesis: S.M. in Engineering and Management, Massachusetts Institute of Technology, Engineering Systems Division, System Design and Management Program, 2019
Cataloged from PDF version of thesis.
Includes bibliographical references (pages 143-147).
Experimentation on course design in MOOCs can determine causal factors that promote learning and can identify aspects of the course where revision is needed. The presence of heterogeneous samples of learners, the difficulty of defining success metrics, and the lack of shared cross-course data are few of the challenges course designers face to evaluate MOOCs. In this thesis, we present a data-driven framework to evaluate design changes in MOOCs. We explore a change from multiple angles -process, proficiency, and perception- and apply various analytical methods -temporal, causal and predictiveto map out the outcome of instruction along multiple dimensions of learning. We demonstrate the application of this framework by evaluating course pacing on a repeated run of a supply chain MOOC by MITx. Self-pacing caused completion rate (-6%), pass rate (-10%), and engagement score (-7%) to drop, although students' satisfaction with course remained unchanged. The impact of pacing on students' outcome was not uniform with some experiencing no change while others encountering a steep fall. The most striking difference was seen in the longitudinal trajectories, with instructor-paced students mostly taking the same trajectory and self-paced students pursuing their own individually paced trajectories. We showed that these trajectories are correlated with student grade, and students with certain characteristics are inclined to pursue a specific trajectory. From these and other observations, we were able to provide practical guidance to course designers on what instructional materials and practices are satisfactory and where change is needed.
by Ahmed Bilal.
S.M. in Engineering and Management
S.M. in Engineering and Management Massachusetts Institute of Technology, Engineering Systems Division, System Design and Management Program
147 pages
eng
Massachusetts Institute of Technology
MIT theses may be protected by copyright. Please reuse MIT thesis content according to the MIT Libraries Permissions Policy, which is available through the URL provided.
http://dspace.mit.edu/handle/1721.1/7582
Engineering Systems Division.
System Design and Management Program.
Using learning analytics to evaluate design changes in MOOCs : a case study on assessing course pacing
Thesis
application/pdf
2022-08-31T16:29:14Z
Publication
Master
Sloan
unknown
unknown
<Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="1746a463-1bfe-4fb1-bdf7-2b6c0dd3ea0c">
<Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type>
<Language>eng</Language>
<Title>Using learning analytics to evaluate design changes in MOOCs : a case study on assessing course pacing</Title>
<PublishedIn>
<Publication>
</Publication>
</PublishedIn>
<PublicationDate>2019</PublicationDate>
<Authors>
<Author>
<DisplayName>Bilal, Ahmed.</DisplayName>
<Affiliation>
<OrgUnit>
</OrgUnit>
</Affiliation>
</Author>
</Authors>
<Editors>
</Editors>
<Publishers>
<Publisher>
<DisplayName>Massachusetts Institute of Technology</DisplayName>
<OrgUnit />
</Publisher>
</Publishers>
<License>http://dspace.mit.edu/handle/1721.1/7582</License>
<Keyword>Engineering Systems Division.</Keyword>
<Keyword>System Design and Management Program.</Keyword>
<Abstract>Experimentation on course design in MOOCs can determine causal factors that promote learning and can identify aspects of the course where revision is needed. The presence of heterogeneous samples of learners, the difficulty of defining success metrics, and the lack of shared cross-course data are few of the challenges course designers face to evaluate MOOCs. In this thesis, we present a data-driven framework to evaluate design changes in MOOCs. We explore a change from multiple angles -process, proficiency, and perception- and apply various analytical methods -temporal, causal and predictiveto map out the outcome of instruction along multiple dimensions of learning. We demonstrate the application of this framework by evaluating course pacing on a repeated run of a supply chain MOOC by MITx. Self-pacing caused completion rate (-6%), pass rate (-10%), and engagement score (-7%) to drop, although students' satisfaction with course remained unchanged. The impact of pacing on students' outcome was not uniform with some experiencing no change while others encountering a steep fall. The most striking difference was seen in the longitudinal trajectories, with instructor-paced students mostly taking the same trajectory and self-paced students pursuing their own individually paced trajectories. We showed that these trajectories are correlated with student grade, and students with certain characteristics are inclined to pursue a specific trajectory. From these and other observations, we were able to provide practical guidance to course designers on what instructional materials and practices are satisfactory and where change is needed.</Abstract>
<Access xmlns="http://purl.org/coar/access_right"
>
</Access>
</Publication>