Transforms for intra prediction residuals based on prediction inaccuracy modeling

Abstract

In intra video coding and image coding, the directional intra prediction is used to reduce spatial redundancy. Intra prediction residuals are encoded with transforms. In this paper, we develop transforms for directional intra prediction residuals. Specifically, we observe that the directional intra prediction is most effective in smooth regions and edges with a particular direction. In the ideal case, edges can be predicted fairly accurately with an accurate prediction direction. In practice, an accurate prediction direction is hard to obtain. Based on the inaccuracy of prediction direction that arises in the design of many practical video coding systems, we can estimate the residual variance and propose a class of transforms based on the estimated variance function. The proposed method is evaluated by the energy compaction property. Experimental results show that with the proposed method, the same amount of energy in directional intra prediction residuals can be preserved with a significantly smaller number of transform coefficients.