Estimating institutional quality with instruments : three essays and applications in education and healthcare

Abstract

1 IsoLATEing: Identifying Counterfactual-specific Treatment Effects with Crossstratum Comparisons -- 2. Leveraging Lotteries for School Value-added: Testing and Estimation -- 3. Estimating Hospital Quality with Quasi-experimental Data.

This thesis develops and applies instrumental variable (IV) techniques to estimate average causal effects in settings with multiple unordered treatments. When an instrument shifts individuals across several distinct margins, classical estimators may not recover causally-interpretable parameters, particularly when effects are heterogeneous and there are fewer instruments than treatment choices. The first chapter of this thesis establishes minimal effect heterogeneity restrictions that permit IV identification of multiple local average treatment effects (LATEs) in such scenarios, using interactions of the instrument and stratifying controls. Under weaker conditional homogeneity assumptions, a novel non-parametric weighting scheme identifies these LATEs. I use this framework to estimate the economic returns to GED certification in a sample that includes individuals who would otherwise obtain a traditional high school diploma, as well as those who would otherwise drop out. The theoretical results may also offer a solution to endogenous attrition bias in randomized trials; I illustrate this in a re-analysis of the 2008 Oregon Health Insurance Experiment. Identification via stratifying interactions may be less feasible with a large number of treatments, though in such settings a researcher may be willing to forego unbiased estimation in favor of a low mean squared error (MSE) across the set of causal estimates. Indeed, policymakers, particularly in the spheres of education and healthcare, often make such trade-offs in forming observational measures of the average effectiveness, or "quality," of the public institutions they regulate. Conventional school value-added models (VAMs), for example, use empirical Bayes methods to shrink school-average test scores towards their grand mean after regression-adjusting for student demographics and past achievement. Under a standard selection-on-observables assumption, these VAMs produce minimum-MSE predictions of true school quality - predictions that increasingly drive school accountability and restructuring policies. The second chapter of this thesis (joint with Joshua Angrist, Parag Pathak, and Christopher Walters), shows how school admission lottery instruments can be used to test VAM validity, quantify the effects of selection bias, and improve conventional school quality predictions. In doing so, we provide a general framework for optimally combining observational and quasi-experimental estimators of multiple treatment effects. An application to Boston middle school quality suggests a failure of selection-on-observables and systematic bias in typical VAM estimates. Nevertheless, we find that the relative magnitude of this selection bias is modest, and policy decisions based on these VAMs are still likely to generate substantial achievement gains. Hybrid quality estimates that incorporate lottery information lead to further reductions in MSE and generate larger gains to students.