2026-09-18T19:34:22Zhttps://dspace.mit.edu/server/oai/request

oai:dspace.mit.edu:1721.1/1637012025-11-18T06:26:45Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023

Kim, Yoon Ghassemi, Marzyeh Puri, Isha Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science 2025-11-17T19:08:08Z 2025-11-17T19:08:08Z 2025-05 2025-08-14T19:33:10.349Z https://hdl.handle.net/1721.1/163701 Large language models (LLMs) have achieved significant performance gains via scaling up model sizes and/or data. However, recent evidence suggests diminishing returns from such approaches, motivating a pivot to scaling test-time compute. Existing deterministic inference-time scaling methods, usually with reward models, cast the task as a search problem, but suffer from a key limitation: early pruning. Due to inherently imperfect reward models, promising trajectories may be discarded prematurely, leading to suboptimal performance. We propose a novel inference-time scaling approach by adapting particle-based Monte Carlo methods. Our method maintains a diverse set of candidates and robustly balances exploration and exploitation. Our empirical evaluation demonstrates that our particle filtering methods have a 4–16x better scaling rate over deterministic search counterparts on both various challenging mathematical and more general reasoning tasks. Using our approach, we show that Qwen2.5-Math-1.5B-Instruct surpasses GPT-4o accuracy in only 4 rollouts, while Qwen2.5-Math-7B-Instruct scales to o1 level accuracy in only 32 rollouts. Our work not only presents an effective method to inference-time scaling, but also connects rich literature in probabilistic inference with inference-time scaling of LLMs to develop more robust algorithms in future work. Code, videos, and further information available at probabilistic-inference-scaling.github.io/ S.M. Massachusetts Institute of Technology In Copyright - Educational Use Permitted Copyright retained by author(s) https://rightsstatements.org/page/InC-EDU/1.0/ Probabilistic Inference for Inference Time Scaling of Language Models Thesis application/pdf Master Master of Science in Electrical Engineering and Computer Science Publication unknown unknown <Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="58504540-07cd-4f65-8c7d-6e1ed6ab9c6d"> <Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type> <Title>Probabilistic Inference for Inference Time Scaling of Language Models</Title> <PublishedIn> <Publication> </Publication> </PublishedIn> <PublicationDate>2025-05</PublicationDate> <Authors> <Author> <DisplayName>Puri, Isha</DisplayName> <Affiliation> <OrgUnit> </OrgUnit> </Affiliation> </Author> </Authors> <Editors> </Editors> <Publishers> <Publisher> <DisplayName>Massachusetts Institute of Technology</DisplayName> <OrgUnit /> </Publisher> </Publishers> <License>https://rightsstatements.org/page/InC-EDU/1.0/</License> <Abstract>Large language models (LLMs) have achieved significant performance gains via scaling up model sizes and/or data. However, recent evidence suggests diminishing returns from such approaches, motivating a pivot to scaling test-time compute. Existing deterministic inference-time scaling methods, usually with reward models, cast the task as a search problem, but suffer from a key limitation: early pruning. Due to inherently imperfect reward models, promising trajectories may be discarded prematurely, leading to suboptimal performance. We propose a novel inference-time scaling approach by adapting particle-based Monte Carlo methods. Our method maintains a diverse set of candidates and robustly balances exploration and exploitation. Our empirical evaluation demonstrates that our particle filtering methods have a 4–16x better scaling rate over deterministic search counterparts on both various challenging mathematical and more general reasoning tasks. Using our approach, we show that Qwen2.5-Math-1.5B-Instruct surpasses GPT-4o accuracy in only 4 rollouts, while Qwen2.5-Math-7B-Instruct scales to o1 level accuracy in only 32 rollouts. Our work not only presents an effective method to inference-time scaling, but also connects rich literature in probabilistic inference with inference-time scaling of LLMs to develop more robust algorithms in future work. Code, videos, and further information available at probabilistic-inference-scaling.github.io/</Abstract> <Access xmlns="http://purl.org/coar/access_right" > </Access> </Publication>