Strong Data Processing Inequalities for Input Constrained Additive Noise Channels
Name
FIcurve.pdf
Description
Submitted version
Size
331.58 KB
Format
Adobe PDF
Checksum (MD5)
bf09e3ecc0294ac5a865a35c613d2be3
Author(s)
Calmon, Flavio du Pin
Polyanskiy, Yury
Wu, Yihong
Date Issued
March 2018
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Calmon, Flavio du Pin, Polyanskiy, Yury and Wu, Yihong. 2018. "Strong Data Processing Inequalities for Input Constrained Additive Noise Channels." 64 (3).
Version
Original manuscript
Abstract
© 1963-2012 IEEE. This paper quantifies the intuitive observation that adding noise reduces available information by means of nonlinear strong data processing inequalities. Consider the random variables $W\to X\to Y$ forming a Markov chain, where $Y = X + Z$ with $X$ and $Z$ real valued, independent and $X$ bounded in $L-{p}$-norm. It is shown that $I(W; Y) \le F-{I}(I(W;X))$ with $F-{I}(t) < t$ whenever $t > 0$ , if and only if $Z$ has a density whose support is not disjoint from any translate of itself. A related question is to characterize for what couplings $(W, X)$ the mutual information $I(W; Y)$ is close to maximum possible. To that end we show that in order to saturate the channel, i.e., for $I(W; Y)$ to approach capacity, it is mandatory that $I(W; X)\to \infty $ (under suitable conditions on the channel). A key ingredient for this result is a deconvolution lemma which shows that postconvolution total variation distance bounds the preconvolution Kolmogorov-Smirnov distance. Explicit bounds are provided for the special case of the additive Gaussian noise channel with quadratic cost constraint. These bounds are shown to be order optimal. For this case, simplified proofs are provided leveraging Gaussian-specific tools such as the connection between information and estimation (I-MMSE) and Talagrand's information-Transportation inequality.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/tit.2017.2782359
