Efficiently Evaluating Targeting Policies: Improving on Champion vs. Challenger Experiments
Name
SSRN-id3017384.pdf
Description
Accepted version
Size
244.79 KB
Format
Adobe PDF
Checksum (MD5)
d0d741d21620b6e460a612a08ab1cf46
Author(s)
Simester, Duncan
Timoshenko, Artem
Zoumpoulis, Spyros I
Date Issued
2020
Journal
Management Science
Publisher
Institute for Operations Research and the Management Sciences (INFORMS)
Version
Author's final manuscript
Abstract
© 2020 INFORMS Champion versus challenger field experiments are widely used to compare the performance of different targeting policies. These experiments randomly assign customers to receive marketing actions recommended by either the existing (champion) policy or the new (challenger) policy, and then compare the aggregate outcomes. We recommend an alternative experimental design and propose an alternative estimation approach to improve the evaluation of targeting policies. The recommended experimental design randomly assigns customers to marketing actions. This allows evaluation of any targeting policy without requiring an additional experiment, including policies designed after the experiment is implemented. The proposed estimation approach identifies customers for whom different policies recommend the same action and recognizes that for these customers there is no difference in performance. This allows for a more precise comparison of the policies. We illustrate the advantages of the experimental design and estimation approach using data from an actual field experiment. We also demonstrate that the grouping of customers, which is the foundation of our estimation approach, can help to improve the training of new targeting policies.
MIT Department
Sloan School of Management
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1287/MNSC.2019.3379
