Decentralized signal processing systems with conservation principles

Abstract

In this thesis, a framework for designing fixed-point and optimization algorithms realized as asynchronous, distributed signal processing systems is developed with an emphasis on the system's stability, robustness, and variational properties. These systems are formed by connecting basic modules together via interconnecting networks. Several classes of systems are constructed using interconnecting networks that obey certain conservation principles where these principles specifically allow steady-state system variables to be interpreted as solutions to optimization problems in a generally non-convex class and provide local conditions on the individual modules to ensure that the variables tend to such solutions, including when the communication between modules is asynchronous and uncoordinated. A particular class of signal processing systems, referred to as scattering systems, is designed that can solve convex and non-convex optimization problems, and where convex problems do not require problem-specific tuning parameters. Connections between scattering systems and their gradient-based and proximal counterparts are also established. The primary contributions of this thesis broadly serve to assist with designing and implementing scattering systems, both by leveraging existing signal processing paradigms and by developing new results in signal processing theory. To demonstrate the utility of the framework, scattering algorithms implemented as web-services and decentralized processor networks are presented and used to solve problems related to optimum filter design, sparse signal recovery, supervised learning, and non-convex regression.