TurKit: Human Computation Algorithms on Mechanical Turk
Author(s)Little, Danny Greg; Chilton, Lydia B.; Goldman, Max; Miller, Robert C.
MetadataShow full item record
Mechanical Turk (MTurk) provides an on-demand source of human computation. This provides a tremendous opportunity to explore algorithms which incorporate human computation as a function call. However, various systems challenges make this difficult in practice, and most uses of MTurk post large numbers of independent tasks. TurKit is a toolkit for prototyping and exploring algorithmic human computation, while maintaining a straight-forward imperative programming style. We present the crash-and-rerun programming model that makes TurKit possible, along with a variety of applications for human computation algorithms. We also present case studies of TurKit used for real experiments across different fields.
DepartmentMassachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology
Association for Computing Machinery
Greg Little, Lydia B. Chilton, Max Goldman, and Robert C. Miller. 2010. TurKit: human computation algorithms on mechanical turk. In Proceedings of the 23nd annual ACM symposium on User interface software and technology (UIST '10). ACM, New York, NY, USA, 57-66. Copyright 2010 ACM
Final published version