Machine learning methods for targeting and new product development

Abstract

Chapter 1: Market research traditionally relies on interviews and focus groups to identify customer needs. User-generated content (UGC), such as online reviews, social media, and call-center data, provides an opportunity to identify customer needs more efficiently. Established methods are not well-suited for large UGC datasets because much of the content is uninformative or repetitive. We propose a machine learning approach for identifying customer needs from UGC and evaluate the method using a new dataset. Once identified, the needs can be used to inform marketing strategy, brand positioning and new product development. Chapter 2: Targeting policies are used in marketing to match different firm actions to different customers. For example, retailers want to send different promotions to different customers, real estate agents want to show different homes, and car dealers want to propose different prices.

We conduct two large-scale field experiments to evaluate seven methods widely used to design targeting policies. The findings compare the performance of the targeting methods and demonstrate how well the methods address common data challenges. The challenges we study are covariate shift, concept shift, information loss through aggregation, and imbalanced data. We show that model-driven methods perform better than distance-driven methods and classification methods when the training data is ideal. However, the performance advantage vanishes in the presence of the challenges that affect the quality of the training data. Chapter 3: Firms typically compare the performance of different targeting policies by implementing the champion versus challenger experimental design. These experiments randomly assign customers to receive marketing actions recommended by either the existing (champion) policy or the new (challenger) policy, and then compare the aggregate outcomes.

We recommend an alternative experimental design and propose an estimation approach to improve the evaluation of targeting policies. The recommended experimental design randomly assigns customers to marketing actions. This allows evaluation of any targeting policy without requiring an additional experiment, including policies designed after the experiment is implemented. The proposed estimation approach identifies customers for whom different policies recommend the same action and recognizes that for these customers there is no difference in performance. This allows for a more precise comparison of the policies. We illustrate the advantages of the experimental design and the estimation approach using data from an actual field experiment. We also demonstrate that the grouping of customers, which is the foundation of our estimation approach, can help to improve the training of new targeting policies.

Chapter 4: Coupon personalization requires to predict how different combinations of coupons affect customer purchasing behavior. We develop a nonparametric model which predicts product choice for the entire assortment of a large retailer. Our model is nonparametric and is based on a deep neural network. The model inputs purchasing histories of individual customers and the coupon assignments to predict individual purchasing decisions. The model operates without ex-ante definitions of product categories. We evaluate the proposed product choice model in simulations. Our model significantly outperforms the baseline machine learning methods in terms of the prediction accuracy. Coupon personalization based on our model also achieves a substantially higher revenue compared to the baseline prediction methods.