Real-Time Prosodic Tennis Commentary via Streaming Vision-Language Models
Name
kelleher-maurakel-meng-eecs-2026-thesis.pdf
Size
35.72 MB
Format
Adobe PDF
Checksum (MD5)
92f853eb26d6441016ea453e276ebc57
Author(s)
Kelleher, Maura
Advisor(s)
Oliva, Aude
Feris, Rogerio
Date Issued
February 2026
Publisher
Massachusetts Institute of Technology
Abstract
Major tennis tournaments feature hundreds of matches across multiple courts, yet human commentators can only cover a fraction of them. This leaves fans without play-by-play coverage for most matches, particularly those on side courts. This thesis presents a system for generating real-time, emotionally expressive AI commentary for tennis matches, addressing both the content generation and prosodic delivery challenges that distinguish engaging sports commentary from robotic narration. We develop two complementary approaches. First, we create a modular pipeline that processes tennis clips through a computer vision module to extract shot metadata, uses a large language model to generate commentary, and synthesizes speech using a prosodic text-to-speech model. This pipeline incorporates an excitement modulation mechanism to control both the linguistic style of the generated text and the prosodic characteristics of the synthesized speech. Second, we work toward a unified approach by fine-tuning StreamingVLM, a vision-language model trained on over 4,000 hours of sports commentary but lacking tennis data. To bridge this gap, we combine commentary from the modular pipeline with a novel scalable pipeline that extracts and cleans commentary from YouTube streams via optical character recognition, automatic speech recognition, and large language model based cleaning. Initial fine-tuning experiments demonstrate that StreamingVLM can learn to generate tennis-specific commentary with correct player identification, appropriate shot descriptions, and accurate score tracking. While limitations remain in commentary cadence, factual accuracy and long-term coherence, these results establish a foundation for further development toward high-quality, emotionally resonant automated tennis commentary.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
