Learning who to target with what via adaptive experimentation to optimize long-term outcomes

Abstract

This paper develops a framework for learning and implementing optimal targeting policies via a sequence of adaptive experiments to maximize long-term customer outcomes. Our framework builds on literature on doubly robust off-policy evaluation and optimization from computer science, statistics, and economics, and can also adapt to potential changes in the environment. We apply our framework to learn optimal discount targeting policies to the current subscribers at Boston Globe to maximize long-term revenue. Since the long-term revenue is not observable, we use intermediate outcomes such as subscribers' short-term revenue and their content consumption to construct a surrogate index and use it to impute the missing long-term revenues. Our method improves the average 1.5-year revenue by $15 and projected 3-year revenue by $40 per subscriber compared to several competitive targeting policies such as a policy that targets no one, a random policy, and a policy that targets subscribers with the highest churn risk. Over a three year period, our approach has a net-positive revenue impact in the range $1.7-$2.8 million compared to the status quo.