TurKit: Human Computation Algorithms on Mechanical Turk
Author(s)
Little, Danny Greg; Chilton, Lydia B.; Goldman, Max; Miller, Robert C.
Downloaduist2010-turkit.pdf (239.1Kb)
OPEN_ACCESS_POLICY
Open Access Policy
Creative Commons Attribution-Noncommercial-Share Alike
Terms of use
Metadata
Show full item recordAbstract
Mechanical Turk (MTurk) provides an on-demand source of human computation. This provides a tremendous opportunity to explore algorithms which incorporate human computation as a function call. However, various systems challenges make this difficult in practice, and most uses of MTurk post large numbers of independent tasks. TurKit is a toolkit for prototyping and exploring algorithmic human computation, while maintaining a straight-forward imperative programming style. We present the crash-and-rerun programming model that makes TurKit possible, along with a variety of applications for human computation algorithms. We also present case studies of TurKit used for real experiments across different fields.
Date issued
2010-10Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer ScienceJournal
UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology
Publisher
Association for Computing Machinery
Citation
Greg Little, Lydia B. Chilton, Max Goldman, and Robert C. Miller. 2010. TurKit: human computation algorithms on mechanical turk. In Proceedings of the 23nd annual ACM symposium on User interface software and technology (UIST '10). ACM, New York, NY, USA, 57-66. Copyright 2010 ACM
Version: Final published version
ISBN
978-1-4503-0271-5