Formal Mathematics via Language-Model Probabilistic Programming
Name
barba-barba-meng-eecs-2026-thesis.pdf
Size
1.74 MB
Format
Adobe PDF
Checksum (MD5)
601a062e0d0c76c82bdf1bc9ff0e6624
Author(s)
Barba da Costa, Mauricio
Advisor(s)
Freer, Cameron
Mansinghka, Vikash
Date Issued
February 2026
Publisher
Massachusetts Institute of Technology
Abstract
Formal mathematics—encompassing tasks such as auto-formalization and theorem proving— offers automated verification signals that make it an attractive domain for language model research. However, existing inference-time strategies for these tasks tend to be inefficient. We apply language model probabilistic programming (LMPP) to formal mathematics, leveraging a framework that allows practitioners to define target distributions over language model outputs through composable potential functions, with inference handled automatically via Sequential Monte Carlo (SMC). We develop multiple potentials that we apply to formal mathematics tasks. We develop two ad-hoc potentials for auto-formalization—incremental type checking and cycle consistency—and two general-purpose ones that we apply specifically to the domain of formal theorem-proving—Long Horizon Temperature Scaling and SMC-based Speculative Decoding. Our empirical results show that these LMPP-based strategies can improve both accuracy and efficiency on formal mathematics benchmarks.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
