An Equivalence Approach to Balance and Placebo Tests
Author(s)Hartman, Erin; Hidalgo, Fernando Daniel
MetadataShow full item record
Recent emphasis on credible causal designs has led to the expectation that scholars justify their research designs by testing the plausibility of their causal identification assumptions, often through balance and placebo tests. Yet current practice is to use statistical tests with an inappropriate null hypothesis of no difference, which can result in equating nonsignificant differences with significant homogeneity. Instead, we argue that researchers should begin with the initial hypothesis that the data are inconsistent with a valid research design, and provide sufficient statistical evidence in favor of a valid design. When tests are correctly specified so that difference is the null and equivalence is the alternative, the problems afflicting traditional tests are alleviated. We argue that equivalence tests are better able to incorporate substantive considerations about what constitutes good balance on covariates and placebo outcomes than traditional tests. We demonstrate these advantages with applications to natural experiments.
DepartmentMassachusetts Institute of Technology. Department of Political Science
American Journal of Political Science
Hartman, Erin and F. Daniel Hidalgo. "An Equivalence Approach to Balance and Placebo Tests." American Journal of Political Science 62, 4 (September 2018): 1000-1013 © 2018 Midwest Political Science Association
Author's final manuscript