Repository logo
Log in(current)
Repository logoMIT Open ScholarshipDSpace@MIT
  1. Home
  2. MIT Open Access Articles
  3. MIT Open Access Articles
  4. SoundNet: learning sound representations from unlabeled video

SoundNet: learning sound representations from unlabeled video

Thumbnail Image
Download
Name

6146-soundnet-learning-sound-representations-from-unlabeled-video.pdf

Description
Published version
Size

5.58 MB

Format

Adobe PDF

Checksum (MD5)

7bd61926f043ef25609d04ec29e3522c

sword-2019-07-11T16:05:58.original.xml (130 B)
Original SWORD entry document
Author(s)
Aytar, Yusuf
•
Vondrick*, Carl
•
Torralba, Antonio
Date Issued
2016
Journal
Advances in Neural Information Processing Systems
Citation
Aytar, Yusuf, Carl Vondrick, and Antonio Torralba, "SoundNet: learning sound representations from unlabeled video." In Lee, D.D., et al., eds., Advances in Neural Information Processing Systems 19 (San Diego, Calif.: Neural Information Processing Systems Foundation, 2016): url doi https://papers.nips.cc/paper/6146-soundnet-learning-sound-representations-from-unlabeled-video ©2016 Author(s)
Version
Final published version
Abstract
We learn rich natural sound representations by capitalizing on large amounts of unlabeled sound data collected in the wild. We leverage the natural synchronization between vision and sound to learn an acoustic representation using two-million unlabeled videos. Unlabeled video has the advantage that it can be economically acquired at massive scales, yet contains useful signals about natural sound. We propose a student-teacher training procedure which transfers discriminative visual knowledge from well established visual recognition models into the sound modality using unlabeled video as a bridge. Our sound representation yields significant performance improvements over the state-of-the-art results on standard benchmarks for acoustic scene/object classification. Visualizations suggest some high-level semantics automatically emerge in the sound network, even though it is trained without ground truth labels. ©2016 Paper presented at the 30th Conference on Neural Information Processing Systems (NeurIPS), Dec. 5-10, 2016, Barcelona, Spain.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
https://hdl.handle.net/1721.1/124993
DOI of Published Version
https://papers.nips.cc/paper/6146-soundnet-learning-sound-representations-from-unlabeled-video
Repository logo
PrivacyPermissionsAccessibilityContact us
Repository logo
Notify us about copyright concerns.