Entropy under additive Bernoulli and spherical noises
Name
isit18_spheremgl.pdf
Description
Accepted version
Size
92.43 KB
Format
Adobe PDF
Checksum (MD5)
8616bb2ba265ffe19fc22a1db887a23f
Author(s)
Ordentlich, Or
Polyanskiy, Yury
Date Issued
June 2018
Journal
2018 IEEE International Symposium on Information Theory (ISIT)
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Ordentlich, Or, and Yury Polyanskiy, "Entropy under additive Bernoulli and spherical noises." 2018 IEEE International Symposium on Information Theory (ISIT 2018) (Piscataway, N.J.: IEEE, 2018): p. 521-25 doi 10.1109/ISIT.2018.8437589 ©2018 Author(s)
Version
Author's final manuscript
Abstract
Let Z[superscript n] be iid Bernoulli (δ) and U[superscript n] be uniform on the set of all binary vectors of weight δ[superscript n] (Hamming sphere). As is well known, the entropies of Z[superscript n] and U[superscript n] are within O(√n). However, if X[superscript n] is another binary random variable independent of Z[superscript n] and U[superscript n], we show that H(X[superscript n]+U[superscript n]) and H(X[superscript n]+Z[superscript n]) are within O(√n) and this estimate is tight. The bound is shown via coupling method. Tightness follows from the observation that the channels x[superscript n]⟼x[superscript n]+U[superscript n] and x[superscript n]⟼x[superscript n]+Z[superscript n] have similar capacities, but the former has zero dispersion. Finally, we show that despite the √n slack in general, the Mrs. Gerber Lemma for H(X[superscript n]+U[superscript n]) holds with only an O(log n) correction compared to its brethren for H(X[superscript n]+Z[superscript n]). ©2019 Paper presented at the 2018 IEEE International Symposium on Information Theory (ISIT 2018), June 17-22, 2018, Vail, Colo.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/ISIT.2018.8437589
