## An exploration of the role of principal inertia components in information theory

##### Author(s)

Calmon, Flavio P.; Varia, Mayank; Medard, Muriel
DownloadMedard_An exploration.pdf (150.9Kb)

OPEN_ACCESS_POLICY

# Open Access Policy

Creative Commons Attribution-Noncommercial-Share Alike

##### Terms of use

##### Metadata

Show full item record##### Abstract

The principal inertia components of the joint distribution of two random variables X and Y are inherently connected to how an observation of Y is statistically related to a hidden variable X. In this paper, we explore this connection within an information theoretic framework. We show that, under certain symmetry conditions, the principal inertia components play an important role in estimating one-bit functions of X, namely f(X), given an observation of Y. In particular, the principal inertia components bear an interpretation as filter coefficients in the linear transformation of p[subscript f(X)|X] into p[subscript f(X)|Y]. This interpretation naturally leads to the conjecture that the mutual information between f(X) and Y is maximized when all the principal inertia components have equal value. We also study the role of the principal inertia components in the Markov chain B → X → Y → B̂, where B and B̂ are binary random variables. We illustrate our results for the setting where X and Y are binary strings and Y is the result of sending X through an additive noise binary channel.

##### Date issued

2014-11##### Department

Lincoln Laboratory; Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science; Massachusetts Institute of Technology. Department of Mathematics; Massachusetts Institute of Technology. Research Laboratory of Electronics##### Journal

Proceedings of the 2014 IEEE Information Theory Workshop (ITW 2014)

##### Publisher

Institute of Electrical and Electronics Engineers (IEEE)

##### Citation

Calmon, Flavio P., Mayank Varia, and Muriel Medard. “An Exploration of the Role of Principal Inertia Components in Information Theory.” 2014 IEEE Information Theory Workshop (ITW 2014) (November 2014).

Version: Original manuscript

##### ISBN

978-1-4799-5999-0

##### ISSN

1662-9019