Searching with Intuition: Exploring (the bounds of) LLM-guided Search with Unknown Objectives
Name
Kaashoek-justinka-SMMR-Management-2025-thesis.pdf
Description
Thesis PDF
Size
6.7 MB
Format
Adobe PDF
Checksum (MD5)
800b64fe73a2c992b8a383c9c59f24a6
Author(s)
Kaashoek, Justin H.
Advisor(s)
Raghavan, Manish
Date Issued
May 2025
Publisher
Massachusetts Institute of Technology
Abstract
Large language models (LLMs) can perform a wide range of search and optimization tasks over discrete spaces. This work seeks to explore the limits of LLM-guided search. We construct a set of text optimization tasks with different levels of "intuitiveness'' and evaluate whether LLMs can effectively optimize objectives. We show that the LLM's performance depends not only on its intuition for the objective, but also on the alignment between the objective and its priors. We also find that the LLM can successfully optimize an objective even without an explicit description of the objective. Our results largely focus on greedy search strategies; we develop a theoretical characterization of conditions under which greedy search is optimal, meaning the LLM's failures result from a fundamental inability to take gradient-like steps, not suboptimal search.
MIT Department
Sloan School of Management
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
