Domain Adaptation of VLM for Soccer Video Understanding
Name
jiang-tjiang25-sm-mba-eecs-sloan-2025_thesis.pdf
Description
Thesis PDF
Size
8.4 MB
Format
Adobe PDF
Checksum (MD5)
993ddb36f7e947efd929bb517409217f
Author(s)
Jiang, Tiancheng(Tony)
Advisor(s)
Zarandi, Mohammad Fazel
Williams, John
Chuang, Isaac
Date Issued
May 2025
Publisher
Massachusetts Institute of Technology
Abstract
Vision Language Models (VLMs) have demonstrated strong performance in multi-modal tasks by effectively aligning visual and textual representations. However, most video under- standing VLM research has been domain-agnostic, leaving the understanding of their transfer learning capability to specialized domains underexplored. In this work, we address this by exploring the adaptability of open-source VLMs to specific domains, and focusing on soccer as an initial case study. Our approach uses large-scale soccer datasets and LLM to create instruction-following data, and use them to iteratively fine-tune the general-domain VLM in a curriculum learning fashion (first teaching the model key soccer concepts to then question answering tasks). The final adapted model, trained using a curated dataset of 20k video clips, exhibits significant improvement in soccer-specific tasks compared to the base model, with a 37.5% relative improvement for the visual question-answering task and an accuracy improvement from 11.8% to 63.5% for the downstream soccer action classification task.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Sloan School of Management
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
