Efficient Seasonal Forecasting of Application Demand with ELF

Abstract

Increased use of data insights to guide ventures have led to an explosion of needs in data services such as data accessibility, mobility, availability, and protection. Particularly in cloud enterprises, this expansion of data services has led to an increased need of AIOps, or intelligent systems that can offer consistent operation while dynamically adjusting their operation for the data services requested. In the field of storage systems, self-management features include proactive management of resources through knowledge of demand and their changing patterns. Previous research on classification, forecasting, trending, and pattern recognition in storage workloads have concluded that there is no universally best predictor for all workload patterns. In addition, these researched methods and their comparisons focus more heavily on accuracy without considering the limitations on overhead and computation power present in a systemoriented approach. This thesis analyzes design tradeoffs and presents ELF, a generic forecasting algorithm of storage workload data that optimizes computation costs in the context of a real-life production system. ELF takes advantage of the fact that the majority of storage workloads possess activity too simple to warrant complex forecasting models. Using a customized classification approach, ELF selects the appropriate predictive model based on the workload’s observed activity and produces accurate forecasts 92 times faster than a generic baseline algorithm while storing 97.5% less data.