Digging Up Threats to Validity: A Data Marshalling Approach to Sensitivity Analysis

Abstract

Causal inference remains a cornerstone for scientific discovery in the natural and social sciences; however, the accuracy of such causal discoveries is susceptible to unobserved confounding bias, the “Achilles heel of most non-experimental studies”. Our principal objective is to bolster the validity of reported causal findings by marshalling pertinent data to corroborate or refute them. In this workshop submission, we describe how data marshalling can turbocharge sensitivity analysis, detail technical challenges, and illustrate a case study as a proof of concept. Our aim in this work is to gather feedback from the audience, gauge interest in solving this open problem relevant to the responsible AI and data management community, and continue iterating on systems that advance the trustworthiness and reproducibility of scientific discoveries.