Imaginative reasoning in probabilistic programs

Abstract

Human reasoning is complex, messy, and approximate, and as a result, has been subject to a millennia-long enterprise to extract principles that are simple, neat, and impeccable. This enterprise is incomplete; there are acts of reasoning that humans perform everyday, often effortlessly, that remain both poorly understood and beyond the capabilities of modern methods of artificial intelligence. Specifically, humans understand the causal structure of the world, and mentally manipulate it to imagine worlds that could have been but were not, and even worlds that could never exist in reality. This thesis investigates computational principles of imaginative reasoning; develops programming languages to express the knowledge upon which imaginative reasoning relies, and upon this foundation introduces practical algorithms of automatic inference. Concretely, we introduce probabilistic programming languages - which encode causal probabilistic models as programs - with two new forms of inference. The first is distributional inference, which means to reason with statistical information rather than observational data. This allows us for instance to address problems of algorithmic fairness, robustness, and perform parameter estimation using data about probabilities, expectations and other distributional properties. The second is causal inference, which allows us in complex simulation models to reason about counterfactual what-if scenarios, as well as causation, i.e., whether some event A is the cause of some other event B. To perform inference, we introduce a number of new algorithms. Unlike traditional methods, these modify the internal structure of the model or reinterpret how it is executed. We introduce parametric inversion, which inverts the causal structure to literally run programs in reverse from observations to causes, and predicate exchange, which relaxes Boolean operators to make inference more tractable. Collectively, these contributions shrink the gap between human and machine reasoning, as well as serve as practical tools for scientific modelling and inference.