Efficient Uncertainty Quantification of Large Language
Models
Name
li-liangela-meng-eecs-2025-thesis.pdf
Description
Thesis PDF
Size
4.04 MB
Format
Adobe PDF
Checksum (MD5)
ca681212bd016468449daa0b4cf6755b
Author(s)
Li, Angela
Advisor(s)
Ghassemi, Marzyeh
Date Issued
September 2025
Publisher
Massachusetts Institute of Technology
Abstract
Large Language Models (LLMs) have demonstrated remarkable success in many applications; however, their reliability remains a critical concern, especially in high-stakes domains such as healthcare, finance, and law. Uncertainty Quantification (UQ) is essential for assessing LLM outputs and ensuring trust. However, existing UQ methods for LLMs face challenges: high computational costs, difficulties in handling unstructured outputs, and limited generalizability. This thesis addresses these challenges by proposing a systematic investigation into robust and efficient UQ methodologies tailored for LLMs. Specifically, this work focuses on: (1) analyzing probing methods to determine how hidden layers encode information relevant to uncertainty and accuracy, (2) developing novel UQ metrics that strongly correlate with actual model performance, and (3) designing computationally efficient pipelines to make UQ practical for real-world applications. By bridging these gaps, this research aims to establish UQ as a reliable tool for evaluating and improving the trustworthiness of LLM outputs, facilitating their safe and effective deployment in critical domains.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
