A Local Characterization for Wyner Common Information
Name
A Local Characterization for Wyner Common Information.pdf
Description
Accepted version
Size
326.85 KB
Format
Adobe PDF
Checksum (MD5)
2ee62f7f508e2c41408ae3e402e82485
Author(s)
Huang, Shao-Lun
Xu, Xiangxiang
Zheng, Lizhong
Wornell, Gregory W
Date Issued
August 2020
Journal
2020 IEEE International Symposium on Information Theory (ISIT)
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Huang, Shao-Lun et al. "A Local Characterization for Wyner Common Information." 2020 IEEE International Symposium on Information Theory, June 2020, Los Angeles, CA, Institute of Electrical and Electronics Engineers, August 2020. © 2020 IEEE
Version
Author's final manuscript
Abstract
While the Hirschfeld-Gebelein-Rényi (HGR) maximal correlation and the Wyner common information share similar information processing purposes of extracting common knowledge structures between random variables, the relationships between these approaches are generally unclear. In this paper, we demonstrate such relationships by considering the Wyner common information in the weakly dependent regime, called ϵ-common information. We show that the HGR maximal correlation functions coincide with the relative likelihood functions of estimating the auxiliary random variables in ϵ-common information, which establishes the fundamental connections these approaches. Moreover, we extend the ϵ-common information to multiple random variables, and derive a novel algorithm for extracting feature functions of data variables regarding their common information. Our approach is validated by the MNIST problem, and can potentially be useful in multi-modal data analyses.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/isit44484.2020.9174206
