Game Theory based Peer Grading Mechanisms for MOOCs
Author(s)Wu, William; Kaashoek, Nicolaas; Tzamos, Christos; Weinberg, Matthew; Daskalakis, Konstantinos
MetadataShow full item record
An efficient peer grading mechanism is proposed for grading the multitude of assignments in online courses. This novel approach is based on game theory and mechanism design. A set of assumptions and a mathematical model is ratified to simulate the dominant strategy behavior of students in a given mechanism. A benchmark function accounting for grade accuracy and workload is established to quantitatively compare effectiveness and scalability of various mechanisms. After multiple iterations of mechanisms under increasingly realistic assumptions, three are proposed: Calibration, Improved Calibration, and Deduction. The Calibration mechanism performs as predicted by game theory when tested in an online crowd-sourced experiment, but fails when students are assumed to communicate. The Improved Calibration mechanism addresses this assumption, but at the cost of more effort spent grading. The Deduction mechanism performs relatively well in the benchmark, outperforming the Calibration, Improved Calibration, traditional automated, and traditional peer grading systems. The mathematical model and benchmark opens the way for future derivative works to be performed and compared.
DepartmentMassachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Proceedings of the Second (2015) ACM Conference on Learning @ Scale (L@S '15)
Association for Computing Machinery (ACM)
William Wu, Constantinos Daskalakis, Nicolaas Kaashoek, Christos Tzamos, and Matthew Weinberg. 2015. Game Theory based Peer Grading Mechanisms for MOOCs. In Proceedings of the Second (2015) ACM Conference on Learning @ Scale (L@S '15). ACM, New York, NY, USA, 281-286.
Author's final manuscript