Information Theoretic Bounds for Distributed Computation Over Networks of Point-to-Point Channels
Name
Dahleh_Information theoretic.pdf
Size
365.76 KB
Format
Adobe PDF
Checksum (MD5)
ec6981d262f1748fc65d32de487c57da
Author(s)
Ayaso, Ola
Shah, Devavrat
Dahleh, Munther A.
Date Issued
December 2010
Journal
IEEE transactions on information theory
Publisher
Institute of Electrical and Electronics Engineers / IEEE Information Theory Society
Citation
Ayaso, O., D. Shah, and M.A. Dahleh. “Information Theoretic Bounds for Distributed Computation Over Networks of Point-to-Point Channels.” Information Theory, IEEE Transactions On 56.12 (2010) : 6020-6039. Copyright © 2010, IEEE
Version
Final published version
Abstract
A network of nodes communicate via point-to-point
memoryless independent noisy channels. Each node has some realvalued
initial measurement or message. The goal of each of the
nodes is to acquire an estimate of a given function of all the initial
measurements in the network. As the main contribution of this
paper, a lower bound on computation time is derived. This bound
must be satisfied by any algorithm used by the nodes to communicate
and compute, so that the mean-square error in the nodes’
estimate is within a given interval around zero. The derivation utilizes
information theoretic inequalities reminiscent of those used
in rate distortion theory along with a novel “perturbation” technique
so as to be broadly applicable. To understand the tightness of
the bound, a specific scenario is considered. Nodes are required to
learn a linear combination of the initial values in the network while
communicating over erasure channels. A distributed quantized algorithm
is developed, and it is shown that the computation time essentially
scales as is implied by the lower bound. In particular, the
computation time depends reciprocally on “conductance”, which is
a property of the network that captures the information-flow bottleneck.
As a by-product, this leads to a quantized algorithm, for
computing separable functions in a network, with minimal computation
time.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/tit.2010.2080850
