An exploration of the role of principal inertia components in information theory
Name
Medard_An exploration.pdf
Size
150.91 KB
Format
Adobe PDF
Checksum (MD5)
7048aca74776ef8bc1f65b81722a095a
Author(s)
Calmon, Flavio P.
Varia, Mayank
Medard, Muriel
Date Issued
November 2014
Journal
Proceedings of the 2014 IEEE Information Theory Workshop (ITW 2014)
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Calmon, Flavio P., Mayank Varia, and Muriel Medard. “An Exploration of the Role of Principal Inertia Components in Information Theory.” 2014 IEEE Information Theory Workshop (ITW 2014) (November 2014).
Version
Original manuscript
Abstract
The principal inertia components of the joint distribution of two random variables X and Y are inherently connected to how an observation of Y is statistically related to a hidden variable X. In this paper, we explore this connection within an information theoretic framework. We show that, under certain symmetry conditions, the principal inertia components play an important role in estimating one-bit functions of X, namely f(X), given an observation of Y. In particular, the principal inertia components bear an interpretation as filter coefficients in the linear transformation of p[subscript f(X)|X] into p[subscript f(X)|Y]. This interpretation naturally leads to the conjecture that the mutual information between f(X) and Y is maximized when all the principal inertia components have equal value. We also study the role of the principal inertia components in the Markov chain B → X → Y → B̂, where B and B̂ are binary random variables. We illustrate our results for the setting where X and Y are binary strings and Y is the result of sending X through an additive noise binary channel.
MIT Department
Lincoln Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology. Department of Mathematics
Massachusetts Institute of Technology. Research Laboratory of Electronics
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/ITW.2014.6970831
