A language-vision model for translation
Name
1192544690-MIT.pdf
Size
2.69 MB
Format
Adobe PDF
Checksum (MD5)
0d7c9035f49649ca48f5cda9ecff5a90
Author(s)
Fu, Allison.
Advisor(s)
Boris Katz.
Date Issued
2020
Publisher
Massachusetts Institute of Technology
Abstract
Machine translation or the automatic translation by computers from a source language to target language is a well-studied, difficult research problem. In recent years, there has been increased interest in grounding translation in vision. We introduce an unsupervised machine translation system grounded in video that can perform Chinese-English translation without the need for a parallel text corpus. In particular, we train separate Chinese and English generative language-vision models on only 267 captioned videos. We then perform translation by sampling video features for an input sentence in Chinese and finding the top-scoring English sentence or translation that describes the sampled video frames. We found that such a system picks out the correct translation with high accuracy and is a promising step towards augmenting language understanding with video.
Description
Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, May, 2020
Cataloged from the official PDF of thesis.
Includes bibliographical references (pages 55-58).
Subjects
Electrical Engineering and Computer Science.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
MIT theses may be protected by copyright. Please reuse MIT thesis content according to the MIT Libraries Permissions Policy, which is available through the URL provided.
Persistent DSpace Link
