The design of mutual information as a global correlation quantifier
Name
entropy-22-00357-v2.pdf
Size
479 KB
Format
Adobe PDF
Checksum (MD5)
ba9f4aabe93394b43206177b5f8b56f1
Author(s)
Carrara, Nicholas
Vanslette, Kevin
Date Issued
March 2020
Journal
Entropy
Publisher
MDPI
Citation
Carrara, Nicholas, and Kevin Vanslette, "The design of mutual information as a global correlation quantifier." Entropy 22, 3 (Mar. 2020): no. 357 doi 10.3390/e22030357 ©2020 Author(s)
Version
Final published version
Abstract
Using first principles from inference, we design a set of functionals for the purposes of ranking joint probability distributions with respect to their correlations. Starting with a general functional, we impose its desired behavior through the Principle of Constant Correlations (PCC), which constrains the correlation functional to behave in a consistent way under statistically independent inferential transformations. The PCC guides us in choosing the appropriate design criteria for constructing the desired functionals. Since the derivations depend on a choice of partitioning the variable space into n disjoint subspaces, the general functional we design is the n-partite information (NPI), of which the total correlation and mutual information are special cases. Thus, these functionals are found to be uniquely capable of determining whether a certain class of inferential transformations, ρ→∗ρ′ , preserve, destroy or create correlations. This provides conceptual clarity by ruling out other possible global correlation quantifiers. Finally, the derivation and results allow us to quantify non-binary notions of statistical sufficiency. Our results express what percentage of the correlations are preserved under a given inferential transformation or variable mapping. Keywords: n-partite information; total correlation; mutual information; entropy; probability theory; correlation
MIT Department
Massachusetts Institute of Technology. Department of Mechanical Engineering
Terms of Use
Creative Commons Attribution 4.0 International license
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.3390/e22030357
