Neural adaptive video streaming with pensieve

Abstract

Client-side video players employ bitrate adaptation algorithms to cater to the ever-growing QoE requirements of users. These ABR algorithms must balance multiple QoE factors, such as maximizing video bitrate and minimizing rebuffering times. Despite the abundance of recently proposed ABR algorithms, state-of-the-art schemes suffer from two practical challenges: (1) throughput prediction is difficult and inaccurate predictions can lead to degraded performance; (2) existing algorithms use fixed heuristics which have been fine-tuned according to strict assumptions about deployment environments - such tuning precludes generalization across network conditions and QoE objectives. To overcome these challenges, we develop Pensieve, a system that generates ABR algorithms entirely using Reinforcement Learning (RL). Pensieve uses RL to train a neural network model that selects bitrates for future video chunks based on observations collected by client video players. Unlike existing approaches, Pensieve does not rely upon pre-programmed models or assumptions about the environment. Instead, it learns to make ABR decisions solely through observations of the resulting performance of past decisions. As a result, Pensieve can automatically learn ABR algorithms that adapt to a wide range of environmental conditions and QoE metrics. We compare Pensieve to state-of-the-art ABR algorithms using trace-driven and real world experiments spanning a wide variety of network conditions, QoE metrics, and video properties. In all considered scenarios, Pensieve outperforms the best stateof- the-art scheme, with improvements in average QoE of 13.1%-25.0%. Pensieve's policies generalize well, outperforming existing schemes even on networks on which it was not trained.