Reduced traces and JITing in Church

Abstract

Church is a Turing-complete probabilistic programming language, designed for inference. By allowing for easy description and manipulation of distributions, it allows one to describe classical Al models in compact ways, providing a language for very rich expression. However, for inference in Bayes nets, Hidden Markov Models, and topic models, the very settings for which probabilistic programming languages like Church were designed for, researchers typically instead write special cased algorithms in regular programming languages to maximize performance. In this paper, we argue that an extremely general language can still support very fast inference. We first introduce the theoretical aspects of our Church-like language, including many implementation tradeoffs. While the language is extremely hands-off and easy to use, we also allow for more detailed specification of the inference. Lastly, we demonstrate empirical results displaying our progress towards general languages which can perform inference quickly, and point out many future directions for research.