Transforms for prediction residuals based on prediction inaccuracy modeling

Abstract

In a typical transform-based image and video compression system, an image or a video frame is predicted from previously encoded information. The prediction residuals are encoded with transforms. With a proper choice of the transform, a large amount of the residual energy compacts into a small number of transform coefficients. This is known as the energy compaction property. Given the covariance function of the signal, the linear transform with the best energy compaction property is the Karhunen Loeve transform. In this thesis, we develop a new set of transforms for prediction residuals. We observe that the prediction process in practical video compression systems is usually not accurate. By studying the inaccuracy of the prediction process, we can derive new covariance functions for prediction residuals. The estimated covariance function is used to generate the Karhunen Loeve transform for residual encoding. In this thesis, we model the prediction inaccuracy for two types of residuals. Specifically, we estimate the covariance function of the directional intra prediction residuals. We show that the covariance function and the optimal transform for directional intra prediction residuals are related with the one-dimensional gradient of boundary predictors. We estimate the covariance function of the motion-compensated prediction residuals. We show that the covariance function and the optimal transform for motion-compensated prediction residuals are related with the two-dimensional gradient of the displaced reference block. The proposed transforms are evaluated using the energy compaction property and the rate-distortion metric in a practical video coding system. Experimental results indicate that the proposed transforms significantly improve the performance in a typical transform-based compression scenario.