2026-09-18T20:14:58Zhttps://dspace.mit.edu/server/oai/request

oai:dspace.mit.edu:1721.1/1635392025-11-06T03:07:20Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023

Wang, Sherrie Zhang, Chenhui Massachusetts Institute of Technology. Institute for Data, Systems, and Society 2025-11-05T19:33:14Z 2025-11-05T19:33:14Z 2025-05 2025-07-16T16:02:30.585Z https://hdl.handle.net/1721.1/163539 https://orcid.org/0000-0003-3915-6099 Large Vision-Language Models (VLMs) have demonstrated impressive performance on complex tasks involving visual input with natural language instructions. However, it remains unclear to what extent capabilities on natural images transfer to Earth observation (EO) data, which are predominantly satellite and aerial images less common in VLM training data. In this work, we propose VLEO-Bench, a comprehensive evaluation framework to quantify the progress of VLMs toward being useful tools for EO data by assessing their abilities on scene understanding, localization and counting, and change detection tasks. Motivated by real-world applications, our framework includes scenarios like urban monitoring, disaster relief, land use, and conservation. We discover that, although state-of-the-art VLMs like GPT-4V possess extensive world knowledge that leads to strong performance on open-ended tasks like location understanding and image captioning, their poor spatial reasoning limits usefulness on object localization and counting tasks. S.M. Massachusetts Institute of Technology Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) Copyright retained by author(s) https://creativecommons.org/licenses/by-nc-nd/4.0/ VLEO-Bench: A Framework to Evaluate Vision-Language Models for Earth Observation Applications Thesis application/pdf Master Master of Science in Social and Engineering Systems Publication unknown unknown <Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="68e8ae68-fb6f-4994-8c31-260cc0545ade"> <Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type> <Title>VLEO-Bench: A Framework to Evaluate Vision-Language Models for Earth Observation Applications</Title> <PublishedIn> <Publication> </Publication> </PublishedIn> <PublicationDate>2025-05</PublicationDate> <Authors> <Author> <DisplayName>Zhang, Chenhui</DisplayName> <Affiliation> <OrgUnit> </OrgUnit> </Affiliation> </Author> </Authors> <Editors> </Editors> <Publishers> <Publisher> <DisplayName>Massachusetts Institute of Technology</DisplayName> <OrgUnit /> </Publisher> </Publishers> <License>https://creativecommons.org/licenses/by-nc-nd/4.0/</License> <Abstract>Large Vision-Language Models (VLMs) have demonstrated impressive performance on complex tasks involving visual input with natural language instructions. However, it remains unclear to what extent capabilities on natural images transfer to Earth observation (EO) data, which are predominantly satellite and aerial images less common in VLM training data. In this work, we propose VLEO-Bench, a comprehensive evaluation framework to quantify the progress of VLMs toward being useful tools for EO data by assessing their abilities on scene understanding, localization and counting, and change detection tasks. Motivated by real-world applications, our framework includes scenarios like urban monitoring, disaster relief, land use, and conservation. We discover that, although state-of-the-art VLMs like GPT-4V possess extensive world knowledge that leads to strong performance on open-ended tasks like location understanding and image captioning, their poor spatial reasoning limits usefulness on object localization and counting tasks.</Abstract> <Access xmlns="http://purl.org/coar/access_right" > </Access> </Publication>