Dynamic load-balancing of StreamIt cluster computations

Abstract

This thesis discusses the design and implementation of a dynamic load-balancing mechanism for computationally distributed programs running on a cluster written in the StreamIt programming language. StreamIt is useful for streaming data applications such as MPEG codecs. The structure of the language carries a lot of static information, such as data rates and computational hierarchy, and therefore lends itself well to parallelization. This work details a simulator for StreamIt cluster computations used to measure metrics such as throughput. Built on top of this simulation is an agent-based market used for load balancing the computation at StreamIt check-points to adapt to exogenously changing loads on the nodes of the cluster. The market models the structure of the computation as a supply chain. Our experiments study the throughput produced by the market compared to other policies, as well as qualitative features such as stability.