Content Moderation and the Formation of Online Communities: A Theoretical Framework

Abstract

We study the impact of content moderation policies in online communities. In our theoretical model, a platform chooses a content moderation policy and individuals choose whether or not to participate in the community according to the fraction of user content that aligns with their preferences. The effects of content moderation, at first blush, might seem obvious: platform speech is restricted. However, when user participation decisions are taken into account, its effects can be more subtle --- and counter-intuitive. For example, our model can straightforwardly demonstrate how moderation policies mayincrease participation and/ordiversify content available on the platform. In our analysis, we explore a rich set of interconnected phenomena related to content moderation in online communities. We first characterize the effectiveness of a natural class of moderation policies for creating and sustaining communities. Building on this, we explore how resource-limited or ideological platforms might set policies, how communities are affected by differing levels of personalization, and competition between platforms. Our model provides a vocabulary and mathematically tractable framework for analyzing platform decisions about content moderation.