Effects of Data Heterogeneity on Distributed Linear System Solvers

Abstract

We focus on the fundamental problem of solving a system of linear equations. In particular, we are interested in distributed linear system solvers, where one taskmaster coordinates any number of workers to attain a solution. There are two predominant and fundamentally different ways of doing this: optimization-based and projection-based solvers. Although there is extensive literature on both classes of algorithms, a rigorous analytical comparison of their performance is lacking. Consequently, there is no concrete understanding of why numerical experiments show that projection-based solvers tend to perform better in many real and synthetic scenarios. In this work, we develop a framework for such analysis, and we use that framework to investigate the comparison of optimization-based and projection-based solvers.