An Efficient Approach to Informative Feature Extraction from Multimodal Data
Name
1811.08979.pdf
Description
Accepted version
Size
1.5 MB
Format
Adobe PDF
Checksum (MD5)
af301b48715b7b682751647dd687b15e
Author(s)
Wang, Lichen
Wu, Jiaxiang
Huang, Shao-Lun
Zheng, Lizhong
Xu, Xiangxiang
Zhang, Lin
Huang, Junzhou
Date Issued
2019
Journal
Proceedings of the AAAI Conference on Artificial Intelligence
Publisher
Association for the Advancement of Artificial Intelligence (AAAI)
Citation
Wang, Lichen, Wu, Jiaxiang, Huang, Shao-Lun, Zheng, Lizhong, Xu, Xiangxiang et al. 2019. "An Efficient Approach to Informative Feature Extraction from Multimodal Data." Proceedings of the AAAI Conference on Artificial Intelligence, 33.
Version
Author's final manuscript
Abstract
Copyright © 2019, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. One primary focus in multimodal feature extraction is to find the representations of individual modalities that are maximally correlated. As a well-known measure of dependence, the Hirschfeld-Gebelein-Rényi (HGR) maximal correlation becomes an appealing objective because of its operational meaning and desirable properties. However, the strict whitening constraints formalized in the HGR maximal correlation limit its application. To address this problem, this paper proposes Soft-HGR, a novel framework to extract informative features from multiple data modalities. Specifically, our framework prevents the “hard” whitening constraints, while simultaneously preserving the same feature geometry as in the HGR maximal correlation. The objective of Soft-HGR is straightforward, only involving two inner products, which guarantees the efficiency and stability in optimization. We further generalize the framework to handle more than two modalities and missing modalities. When labels are partially available, we enhance the discriminative power of the feature representations by making a semi-supervised adaptation. Empirical evaluation implies that our approach learns more informative feature mappings and is more efficient to optimize.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1609/AAAI.V33I01.33015281
