Minimalist Approach to End-to-End Vision Language
Navigation with Multi-Modal Foundation Model Features
Name
mishra-mk314k-meng-eecs-2025-thesis.pdf
Description
Thesis PDF
Size
19.37 MB
Format
Adobe PDF
Checksum (MD5)
ece5e7d52ad839a57f73ae7ffc45d7f4
Author(s)
Mishra, Kartikesh
Advisor(s)
Rus, Daniela
Date Issued
May 2025
Publisher
Massachusetts Institute of Technology
Abstract
Recent vision-language navigation (VLN) approaches leverage large models, prompt engineering, and/or explicit reasoning for instruction interpretation and agent guidance. We introduce MiniNav, a minimalist framework employing frozen vision-language foundation models as patch-wise feature extractors, avoiding data and compute heavy fine-tuning and cumbersome language model reasoning. Our lightweight control policies (∼ 10⁵ trainable parameters) are trained on a compact dataset of language-based specified navigational behaviors (∼ 10² runs, ∼ 10⁴ frames per behavior). We demonstrate generalization to novel objects and scenes, including direct real-world transfer, despite training on only two objects in a single simulated environment. Through its simple and scalable design, MiniNav provides an alternative to computationally intensive pipelines for robust real-world instruction-following. Our solution can provide a reference for evaluating the effective edge of more complex and larger VLN policies.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link