Evaluating Large Language Models as Circuit Design Assistants
Name
cox-coxm-meng-eecs-2024-thesis.pdf
Description
Thesis PDF
Size
1.19 MB
Format
Adobe PDF
Checksum (MD5)
501b4ffbfc839393d8fb36d932d3dfcf
Author(s)
Cox, Matthew J.
Advisor(s)
Han, Ruonan
Date Issued
September 2024
Publisher
Massachusetts Institute of Technology
Abstract
Large language models (LLMs) have exploded in capability in recent years. Previous attempts at AI systems for circuit design have had limited proficiency and been restricted in problem scope. LLMs, with their breadth of knowledge and reasoning ability, are a promising technology for a much more general-purpose circuit design assistant. We developed a dataset of electrical engineering problems and solutions with which to test an LLM-based system, since no such publicly available dataset exists to our knowledge; unmodified GPT-4 was able to solve 42% of the problems. We did a preliminary comparison of several knowledge bases to use for RAG knowledge injection, finding that a small, curated set of resources performed better than a larger, less-focused set of resources, though there were confounding factors which may have skewed the result. While this work is a start, significant future work is needed to continue developing an LLM-based circuit design assistant.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)
Copyright retained by author(s)
Persistent DSpace Link
