Strong Data-Processing Inequalities for Channels and Bayesian Networks
Name
1508.06025.pdf
Description
Accepted version
Size
392.18 KB
Format
Adobe PDF
Checksum (MD5)
7a3865cb8b602cd20fe6f09ba0a04a75
Author(s)
Polyanskiy, Yury
Wu, Yihong
Date Issued
April 2017
Journal
Convexity and Concentration
Publisher
Springer
Citation
Polyanskiy, Yury and Yihong Wu. "Strong Data-Processing Inequalities for Channels and Bayesian Networks." Convexity and Concentration, IMA Volumes in Mathematics and its Applications, 161, Springer, 2017, 211-249. © 2017 Springer Science+Business Media LLC
Version
Author's final manuscript
Abstract
The data-processing inequality, that is, I(U; Y ) ≤ I(U; X) for a Markov chain U → X → Y, has been the method of choice for proving impossibility (converse) results in information theory and many other disciplines. Various channel-dependent improvements (called strong data-processing inequalities, or SDPIs) of this inequality have been proposed both classically and more recently. In this note we first survey known results relating various notions of contraction for a single channel. Then we consider the basic extension: given SDPI for each constituent channel in a Bayesian network, how to produce an end-to-end SDPI?
Our approach is based on the (extract of the) Evans-Schulman method, which is demonstrated for three different kinds of SDPIs, namely, the usual Ahlswede-Gács type contraction coefficients (mutual information), Dobrushin’s contraction coefficients (total variation), and finally the F I -curve (the best possible non-linear SDPI for a given channel). Resulting bounds on the contraction coefficients are interpreted as probability of site percolation. As an example, we demonstrate how to obtain SDPI for an n-letter memoryless channel with feedback given an SDPI for n = 1.
Finally, we discuss a simple observation on the equivalence of a linear SDPI and comparison to an erasure channel (in the sense of “less noisy” order). This leads to a simple proof of a curious inequality of Samorodnitsky (2015), and sheds light on how information spreads in the subsets of inputs of a memoryless channel.
Description
Part of the The IMA Volumes in Mathematics and its Applications book series (IMA, volume 161).
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1007/978-1-4939-7005-6_7
