Inference Plans for Hybrid Probabilistic Inference

Abstract

Advanced probabilistic programming languages (PPLs) use hybrid inference systems to combine symbolic exact inference and Monte Carlo sampling to improve inference performance. These systems use heuristics to partition random variables within the program into variables that are represented symbolically and variables that are represented by sampled values, and in general, they make no guarantee that the partitioning is optimal. In this thesis, I present inference plans, a programming interface that enables developers to choose a specific partitioning of random variables during hybrid inference. I further present Siren, a new PPL that enables developers to use annotations to specify inference plans. To assist developers with statically reasoning about whether an inference plan can be implemented, I present an abstract-interpretation-based static analysis for Siren for determining inference plan satisfiability, and prove the analysis is sound with respect to Siren's semantics. In our evaluation, the results show that custom inference plans can produce up to ~1000x better accuracy compared to the default heuristics. They further show that the static analysis is precise in practice, identifying all satisfiable inference plans in 6 out of 7 benchmarks.