Notice
This is not the latest version of this item. The latest version can be found at:https://dspace.mit.edu/handle/1721.1/138075.2
Self-Supervised Audio-Visual Co-Segmentation
dc.contributor.author | Rouditchenko, Andrew | |
dc.contributor.author | Zhao, Hang | |
dc.contributor.author | Gan, Chuang | |
dc.contributor.author | McDermott, Josh | |
dc.contributor.author | Torralba, Antonio | |
dc.date.accessioned | 2021-11-09T21:16:55Z | |
dc.date.available | 2021-11-09T21:16:55Z | |
dc.date.issued | 2019-05 | |
dc.identifier.uri | https://hdl.handle.net/1721.1/138075 | |
dc.description.abstract | © 2019 IEEE. Segmenting objects in images and separating sound sources in audio are challenging tasks, in part because traditional approaches require large amounts of labeled data. In this paper we develop a neural network model for visual object segmentation and sound source separation that learns from natural videos through self-supervision. The model is an extension of recently proposed work that maps image pixels to sounds [1]. Here, we introduce a learning approach to disentangle concepts in the neural networks, and assign semantic categories to network feature channels to enable independent image segmentation and sound source separation after audio-visual training on videos. Our evaluations show that the disentangled model outperforms several baselines in semantic segmentation and sound source separation. | en_US |
dc.language.iso | en | |
dc.publisher | IEEE | en_US |
dc.relation.isversionof | 10.1109/icassp.2019.8682467 | en_US |
dc.rights | Creative Commons Attribution-Noncommercial-Share Alike | en_US |
dc.rights.uri | http://creativecommons.org/licenses/by-nc-sa/4.0/ | en_US |
dc.source | arXiv | en_US |
dc.title | Self-Supervised Audio-Visual Co-Segmentation | en_US |
dc.type | Article | en_US |
dc.identifier.citation | Rouditchenko, Andrew, Zhao, Hang, Gan, Chuang, McDermott, Josh and Torralba, Antonio. 2019. "Self-Supervised Audio-Visual Co-Segmentation." | |
dc.contributor.department | MIT-IBM Watson AI Lab | en_US |
dc.eprint.version | Author's final manuscript | en_US |
dc.type.uri | http://purl.org/eprint/type/ConferencePaper | en_US |
eprint.status | http://purl.org/eprint/status/NonPeerReviewed | en_US |
dc.date.updated | 2019-10-03T13:03:40Z | |
dspace.date.submission | 2019-10-03T13:03:45Z | |
mit.metadata.status | Authority Work and Publication Information Needed | en_US |