Essays on Mechanisms Underlying Belief Updating with Applications in Wisdom of Crowds

Abstract

This dissertation consists of three chapters on understanding how people update their beliefs after learning about others’ opinions and how we can leverage belief-updating to improve Wisdom of Crowds.

In Chapter One, I propose a new Revealed Expertise (RE) algorithm that uses the “RE measure”, which is a scaled amount of belief updating given numerical advice (i.e., the group mean), as a proxy for prior variance to better reflect the relative expertise of each agent in a crowd. The intuition, which I confirm both theoretically and empirically, is that those who are less swayed by the group mean tend to be more accurate in their initial judgment. Therefore, using inverse-variance weighting with the RE measures as the variance inputs outperforms the existing wisdom-of-crowds methods by over-weighting the more accurate initial judgments in the aggregation. Crucially, I demonstrate that while self-reported confidence reflects one’s feeling of uncertainty given one’s available information, advice-taking reveals the amount of information one has and has not taken into account in their initial judgment. Therefore, the RE algorithm is able to successfully identify the experts, even when self-reported confidence fails.

In Chapter Two, I develop a boundedly rational model to characterize the relationship among stated confidence, uncertainty, expertise, and advice-taking. The semi-Bayesian belief-updating model I develop is able to reconcile two important empirical phenomena. First, I demonstrate that even though agents can state a high confidence (i.e., low first-order uncertainty), they may put a large weight on the advice in belief-updating if their estimate of their stated confidence is imprecise (i.e., large second-order uncertainty due to their lack of information). Second, I show that the distance effect (i.e., the weight on advice tends to decrease as the distance between the initial estimate and the advice increases), a widely documented empirical pattern in advice-taking, can be a consequence of people updating their beliefs following a semi-Bayesian updating heuristics given their cognitive limitation.

In Chapter Three, I propose an experimental paradigm to examine the role of (preference-based) motivated reasoning in biased advice-taking. In an incentivized task assessing the accuracy of nonpolitical news headlines, we find partisan bias in advice-taking. We then adjudicate between two possible mechanisms for this biased advice-taking: a preference-based account, where participants 3 are motivated to take less advice from counter-partisans because doing so is unpleasant; versus a belief-based account, where participants sincerely believe co-partisans are more competent at the task (even though this belief is incorrect). To do so, we examine the impact of a substantial increase in the stakes, which should increase accuracy motivations (and thereby reduce the relative impact of partisan motivations). We find that increasing the stakes does not reduce biased advice-taking, hence no evidence to support the bias is driven by preference. Instead, in two follow-up experiments, we show evidence of the belief-based account being the main driver of the biased advice-taking.