Game Theory based Peer Grading Mechanisms for MOOCs
Name
Daskalakis_Game theory.pdf
Size
736.48 KB
Format
Adobe PDF
Checksum (MD5)
65111b7d0cd8ec5ec6f1a06b56dc1f16
Author(s)
Wu, William
Kaashoek, Nicolaas
Tzamos, Christos
Weinberg, Matthew
Daskalakis, Konstantinos
Date Issued
March 2015
Journal
Proceedings of the Second (2015) ACM Conference on Learning @ Scale (L@S '15)
Publisher
Association for Computing Machinery (ACM)
Citation
William Wu, Constantinos Daskalakis, Nicolaas Kaashoek, Christos Tzamos, and Matthew Weinberg. 2015. Game Theory based Peer Grading Mechanisms for MOOCs. In Proceedings of the Second (2015) ACM Conference on Learning @ Scale (L@S '15). ACM, New York, NY, USA, 281-286.
Version
Author's final manuscript
Abstract
An efficient peer grading mechanism is proposed for grading the multitude of assignments in online courses. This novel approach is based on game theory and mechanism design. A set of assumptions and a mathematical model is ratified to simulate the dominant strategy behavior of students in a given mechanism. A benchmark function accounting for grade accuracy and workload is established to quantitatively compare effectiveness and scalability of various mechanisms. After multiple iterations of mechanisms under increasingly realistic assumptions, three are proposed: Calibration, Improved Calibration, and Deduction. The Calibration mechanism performs as predicted by game theory when tested in an online crowd-sourced experiment, but fails when students are assumed to communicate. The Improved Calibration mechanism addresses this assumption, but at the cost of more effort spent grading. The Deduction mechanism performs relatively well in the benchmark, outperforming the Calibration, Improved Calibration, traditional automated, and traditional peer grading systems. The mathematical model and benchmark opens the way for future derivative works to be performed and compared.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/2724660.2728676
