Theory Instead of Experiment (TIE): A Creator Valuation System at Tencent

Abstract

Experiments are informative but should be used judiciously as a costly resource. Well-constructed theory may serve as a substitute. We develop a ''Theory Instead of Experiment'' (TIE) framework and, in collaboration with Tencent, apply the framework to assess how much value (e.g., user clicks) each creator contributes to its WeChat Official Accounts Platform. This TIE application models content demand and supply upon the counterfactual departure of a creator. The demand model predicts user clicks based on estimated user preferences, while the supply model captures the platform's content distribution response. Together, they predict how each creator influences user engagement through the platform's content distribution strategy. We test the predictions of the TIE system with 168 experiments, each examining a different mix of creators and involving more than 9 million unique users. The TIE system and the experiments demonstrate a 97% correlation on the key performance metric (change in user clicks). Based on its low costs, high accuracy, granular output, and minimal latency, Tencent has deployed the TIE system as the default approach to creator valuation, assessing tens of millions of creators each day while avoiding a 2.5% user click loss associated with a typical experiment.