Figuring Figures: An assessment of large language models on different modalities of math word problems

Abstract

This paper presents a new dataset of geometry word problems in three forms: with figures, with code that produces these figures, and purely textual. Having versions of the same question which use different modalities allows for a more direct comparison of the performance of machine learning models on mathematical question answering across different modalities of input. We evaluate several multi-modal large language models and find they consistently perform best on the plain text descriptions and worst on the version with images.