Validating Bayesian truth serum in large-scale online human experiments
Name
journal.pone.0177385.pdf
Size
2.24 MB
Format
Adobe PDF
Checksum (MD5)
65acb5e7c96afe644410489a80f59e80
Author(s)
Pickard, Galen
Frank, Morgan Ryan
Cebrian, Manuel
Rahwan, Iyad
Date Issued
May 2017
Journal
PLOS ONE
Publisher
Public Library of Science
Citation
Frank, Morgan R. et al. “Validating Bayesian Truth Serum in Large-Scale Online Human Experiments.” Edited by Chuhsing Kate Hsiao. PLOS ONE 12, 5 (May 2017): e0177385 © 2017 Frank et al
Version
Final published version
Abstract
This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Bayesian truth serum (BTS) is an exciting new method for improving honesty and information quality in multiple-choice survey, but, despite the method's mathematical reliance on large sample sizes, existing literature about BTS only focuses on small experiments. Combined with the prevalence of online survey platforms, such as Amazon's Mechanical Turk, which facilitate surveys with hundreds or thousands of participants, BTS must be effective in large-scale experiments for BTS to become a readily accepted tool in real-world applications. We demonstrate that BTS quantifiably improves honesty in large-scale online surveys where the "honest" distribution of answers is known in expectation on aggregate. Furthermore, we explore a marketing application where "honest" answers cannot be known, but find that BTS treatment impacts the resulting distributions of answers.
MIT Department
Massachusetts Institute of Technology. Media Laboratory
Terms of Use
Creative Commons Attribution 4.0 International License
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1371/JOURNAL.PONE.0177385
