Causal Inference: Heterogeneous Effects and Non-stationary Environments

Abstract

The capability of large businesses and eCommerce platforms to utilize vast amounts of customer data has unlocked the possibility of using advanced analytics methods to customize marketing strategies. We consider the stage of conversion in a marketing funnel, where a customer has arrived on the platform and chooses whether to purchase one of the options offered. In this thesis, we present two lines of work that address the question of whether showing more options improves purchase probability on two levels: population and individual. Results are centered around data from a field experiment run by an online platform. In the setting of the experiment, the population and individual level effects can be understood through the lens of causal inference and the estimation of treatment effects.

First, we use the experiment data to build causal models that aim to maximize the probability of purchase by customizing the number of options shown to each customer. We show that even when advanced analytics and careful model selection procedures are used, the produced models can fail to generalize well to new data. We conclude this first section by showing strong evidence that the I.I.D. assumption, fundamental for generalization of machine learning models, is violated for the data we consider.

In the second part, we address the problem of estimating treatment effects in non-stationary data. In this setting, using old data to make inferences can lead to unreliable results. We propose a novel procedure that helps smooth out the data non-stationarity by providing a way to resample previous data to match the distribution in the current time period. We demonstrate its effectiveness on the experiment data and conduct sensitivity analysis. Finally, we validate the procedure through experiments on simulated data.