2026-09-20T22:56:22Zhttps://dspace.mit.edu/server/oai/requestoai:dspace.mit.edu:1721.1/1448732022-08-30T03:51:10Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023
Oliva, Aude
Agarwal, Anisha
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
2022-08-29T16:17:43Z
2022-08-29T16:17:43Z
2022-05
2022-05-27T16:18:36.566Z
https://hdl.handle.net/1721.1/144873
In this thesis, we re-implement previous work exploring image to speech captioning. We expand upon the work to implement video to speech captioning. Specifically, we implement a text-free image to speech captioning pipeline that integrates four distinct machine learning models. We alter the models to process video data rather than image data and analyze the resulting speech captions. We conduct experiments on the Wav2Vec2 and HuBERT Automatic Speech Recognition models, and identify which works best with synthesized speech.
M.Eng.
Massachusetts Institute of Technology
In Copyright - Educational Use Permitted
Copyright MIT
http://rightsstatements.org/page/InC-EDU/1.0/
Text-Free Audio Captions of Short Videos from Latent Space Representation
Thesis
application/pdf
Master
Master of Engineering in Electrical Engineering and Computer Science
Publication
unknown
unknown
<Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="3b7bb414-d55f-499a-bade-1b4145cee308">
<Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type>
<Title>Text-Free Audio Captions of Short Videos from Latent Space Representation</Title>
<PublishedIn>
<Publication>
</Publication>
</PublishedIn>
<PublicationDate>2022-05</PublicationDate>
<Authors>
<Author>
<DisplayName>Agarwal, Anisha</DisplayName>
<Affiliation>
<OrgUnit>
</OrgUnit>
</Affiliation>
</Author>
</Authors>
<Editors>
</Editors>
<Publishers>
<Publisher>
<DisplayName>Massachusetts Institute of Technology</DisplayName>
<OrgUnit />
</Publisher>
</Publishers>
<License>http://rightsstatements.org/page/InC-EDU/1.0/</License>
<Abstract>In this thesis, we re-implement previous work exploring image to speech captioning. We expand upon the work to implement video to speech captioning. Specifically, we implement a text-free image to speech captioning pipeline that integrates four distinct machine learning models. We alter the models to process video data rather than image data and analyze the resulting speech captions. We conduct experiments on the Wav2Vec2 and HuBERT Automatic Speech Recognition models, and identify which works best with synthesized speech.</Abstract>
<Access xmlns="http://purl.org/coar/access_right"
>
</Access>
</Publication>