Application workload prediction and placement in cloud computing systems

Abstract

Cloud computing has become popular in recent years. Companies such as Amazon and Microsoft host large datacenters of networked machines available for users to rent. These users are varied: from individual researchers to large companies. Their workloads range from short, resource-intensive jobs to long-running user-facing services. As cloud environments become more heavily used, provisioning the underlying network becomes more important. Previous approaches to deal with this problem involve changing the network infrastructure, for instance by imposing a particular topology [34] or creating a new routing protocol [27]. While these techniques are interesting and successful in their own right, we ask a different question: How can we improve cloud networks without changing the network itself? This question is motivated by two desires: first, that customers be able to improve their application's performance without necessarily involving the provider, and second, that our techniques be immediately applicable to today's cloud networks. This dissertation presents an end-to-end system, Cicada, which improves application performance on cloud networks. Cicada tackles two problems: how to model and predict an application's workload, and how to place applications on machines in the network. Cicada can be used by either cloud providers or customers. When used by a cloud provider, Cicada enables the provider to offer certain network performance guarantees to its customers. These guarantees give customers the confidence to use cloud resources when building their own user-facing applications (as there is no longer a risk of the cloud under-provisioning for the customer's network needs), and allow providers to better utilize their network. When used by customers, Cicada enables customers to satisfy their own objectives, such as minimizing the completion time of their application. To do this, Cicada exploits variation in the underlying cloud network to use the fastest paths most frequently. This requires an extension to Cicada, called Choreo, which performs quick, accurate, client-side measurement. We evaluate Cicada using data we collected from HP Cloud, a deployed network with real users. Cicada's workload prediction algorithm outperforms the existing state-of-the-art [20] by up to 90%. Its placement method reduces application completion time by an average of 22%-43%(maximum improvement: 79%) when applications arrive in real-time, and doubles inter-rack utilization in certain datacenter topologies. These results and others herein corroborate our thesis that application performance can be improved in cloud networks without making any changes to the network itself.