Entropy under additive Bernoulli and spherical noises
Author(s)
Ordentlich, Or; Polyanskiy, Yury
DownloadAccepted version (92.42Kb)
Terms of use
Metadata
Show full item recordAbstract
Let Z[superscript n] be iid Bernoulli (δ) and U[superscript n] be uniform on the set of all binary vectors of weight δ[superscript n] (Hamming sphere). As is well known, the entropies of Z[superscript n] and U[superscript n] are within O(√n). However, if X[superscript n] is another binary random variable independent of Z[superscript n] and U[superscript n], we show that H(X[superscript n]+U[superscript n]) and H(X[superscript n]+Z[superscript n]) are within O(√n) and this estimate is tight. The bound is shown via coupling method. Tightness follows from the observation that the channels x[superscript n]⟼x[superscript n]+U[superscript n] and x[superscript n]⟼x[superscript n]+Z[superscript n] have similar capacities, but the former has zero dispersion. Finally, we show that despite the √n slack in general, the Mrs. Gerber Lemma for H(X[superscript n]+U[superscript n]) holds with only an O(log n) correction compared to its brethren for H(X[superscript n]+Z[superscript n]). ©2019 Paper presented at the 2018 IEEE International Symposium on Information Theory (ISIT 2018), June 17-22, 2018, Vail, Colo.
Date issued
2018-06Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer ScienceJournal
2018 IEEE International Symposium on Information Theory (ISIT)
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Ordentlich, Or, and Yury Polyanskiy, "Entropy under additive Bernoulli and spherical noises." 2018 IEEE International Symposium on Information Theory (ISIT 2018) (Piscataway, N.J.: IEEE, 2018): p. 521-25 doi 10.1109/ISIT.2018.8437589 ©2018 Author(s)
Version: Author's final manuscript
ISBN
978-1-5386-4780-6