Message quantization in belief propagation: Structural results in the low-rate regime

Abstract

Motivated by distributed inference applications in unreliable communication networks, we adapt the popular (sum-product) belief propagation (BP) algorithm under the constraint of discrete-valued messages. We show that, in contrast to conventional BP, the optimal message-generation rules are node-dependent and iteration-dependent, each rule making explicit use of local memory from all past iterations. These results expose both the intractability of optimal design and an inherent structure that can be exploited for tractable approximate design. We propose one such approximation and demonstrate its efficacy on canonical examples. We also discuss extensions to communication networks with lossy links (e.g., erasures) or topologies that differ from the graph underlying the probabilistic model.