Adaptive multiple description mode selection for error resilient video communications

Abstract

Streaming video applications must be able to withstand the potentially harsh conditions present on best-effort networks like the Internet, including variations in available bandwidth, packet losses, and delay. Multiple description (MD) video coding is one approach that can be used to reduce the detrimental effects caused by transmission over best-effort networks. In a multiple description system, a video sequence is coded into two or more complementary streams in such a way that each stream is independently decodable. The quality of the received video improves with each received description, and the loss of any one of these descriptions does not cause complete failure. A number of approaches have been proposed for MD coding, where each provides a different tradeoff between compression efficiency and error resilience. How effectively each method achieves this tradeoff depends on network conditions as well as on the characteristics of the video itself. This thesis proposes an adaptive MD coding approach that adapts to changing conditions through the use of MD mode selection. The encoder in this system is able to accurately estimate the expected end-to-end distortion, accounting for both compression and packet-loss-induced distortions, as well as for the bursty nature of channel losses and the effective use of multiple transmission paths.

(cont.) With this model of the expected end-to-end distortion, the encoder selects between MD coding modes in a rate-distortion (R-D) optimized manner to most effectively trade-off compression efficiency for error resilience. We show how this approach adapts to both the local characteristics of the video and to network conditions and demonstrate the resulting gains in performance using an H.264-based adaptive MD video coder. We also analyze the sensitivity of this system to imperfect knowledge of channel conditions and explore the benefits of using such a system with both single and multiple paths.