Reliable and Generalizable Real-World Planning with LLM-based Formalized Programming

Abstract

While large language models (LLMs) have recently demonstrated strong potential in solving planning problems, LLMs, as zero-shot planners themselves, are still not capable of directly generating valid plans for complex planning problems such as multi-constraint or long-horizon tasks. This motivates the needs to develop a robust and reliable planning system for complex real-world planning problems. Furthermore, many frameworks aiming to solve complex planning problems often rely on task-specific preparatory efforts, such as task-specific in-context examples and pre-defined critics or verifiers, which limits their cross-task generalization capability. This motivates the needs to extend the robust and reliable planning systems to have strong generalization capability. In this thesis, we first develop an LLM-based planning framework that formalizes and solves complex multi-constraint planning problems as constrained satisfiability problems and can reliably identify the unsatisfiable cores for unsatisfiable requirements, provide failure reasons, and offers personalized modification suggestions. Then, we generalize the paradigm by proposing a general-purpose framework that leverages LLMs to capture key information from planning problems and formally formulate and solve them as optimization problems from scratch, with no task-specific examples needed. Comprehensive experimental results have shown that our frameworks significantly outperform the baselines and have strong performance across tasks and LLMs.