Figuring Figures: An assessment of large language models on different modalities of math word problems
Name
3674029.3674041.pdf
Size
480.21 KB
Format
Adobe PDF
Checksum (MD5)
c5c34d8833e09ef5468eb14df7511378
Author(s)
Wang, Yan
Lynch, Jayson
Krueger, Elizabeth
Date Issued
May 24, 2024
Publisher
ACM|2024 9th International Conference on Machine Learning Technologies (ICMLT)
Citation
Wang, Yan, Lynch, Jayson and Krueger, Elizabeth. 2024. "Figuring Figures: An assessment of large language models on different modalities of math word problems."
Version
Final published version
Abstract
This paper presents a new dataset of geometry word problems in three forms: with figures, with code that produces these figures, and purely textual. Having versions of the same question which use different modalities allows for a more direct comparison of the performance of machine learning models on mathematical question answering across different modalities of input. We evaluate several multi-modal large language models and find they consistently perform best on the plain text descriptions and worst on the version with images.
Description
ICMLT 2024, May 24–26, 2024, Oslo, Norway
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3674029.3674041
