An exploration of the role of principal inertia components in information theory
Author(s)Calmon, Flavio P.; Varia, Mayank; Medard, Muriel
MetadataShow full item record
The principal inertia components of the joint distribution of two random variables X and Y are inherently connected to how an observation of Y is statistically related to a hidden variable X. In this paper, we explore this connection within an information theoretic framework. We show that, under certain symmetry conditions, the principal inertia components play an important role in estimating one-bit functions of X, namely f(X), given an observation of Y. In particular, the principal inertia components bear an interpretation as filter coefficients in the linear transformation of p[subscript f(X)|X] into p[subscript f(X)|Y]. This interpretation naturally leads to the conjecture that the mutual information between f(X) and Y is maximized when all the principal inertia components have equal value. We also study the role of the principal inertia components in the Markov chain B → X → Y → B̂, where B and B̂ are binary random variables. We illustrate our results for the setting where X and Y are binary strings and Y is the result of sending X through an additive noise binary channel.
DepartmentLincoln Laboratory; Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science; Massachusetts Institute of Technology. Department of Mathematics; Massachusetts Institute of Technology. Research Laboratory of Electronics
Proceedings of the 2014 IEEE Information Theory Workshop (ITW 2014)
Institute of Electrical and Electronics Engineers (IEEE)
Calmon, Flavio P., Mayank Varia, and Muriel Medard. “An Exploration of the Role of Principal Inertia Components in Information Theory.” 2014 IEEE Information Theory Workshop (ITW 2014) (November 2014).