Algorithms for transform selection in multiple-transform video compression

Abstract

Selecting proper transforms for video compression has been based on the rate-distortion criterion. Transforms that appear reasonable are incorporated into a video coding system and their performance is evaluated. This approach is tedious when a large number of transforms are used. A quick approach to evaluate these transforms is based on the energy compaction property. With a proper transform, an image or motion-compensated residual can be represented quite accurately with a small fraction of the transform coefficients. This is referred to as the energy compaction property. However, when multiple transforms are used, selecting the best transform for each block that leads to the best energy compaction is difficult. In this thesis, we develop two algorithms to solve this problem. The first algorithm, which is computationally simple, leads to a locally optimal solution. The second algorithm, which is more intensive computationally, gives a globally optimal solution. We provide a detailed discussion on the ideas and steps of the algorithms, followed by the theoretical analysis of the performance. We verify that these algorithms are useful in a practical setting, by comparing and showing the consistency with rate-distortion results from previous research. We apply the algorithms when a large number of transforms are used. These transforms are equal-length 1D-DCTs in 4x4 blocks, which try to characterize as many 1D structures as possible in motion-compensation residuals. By evaluating the energy compaction property of up to 245 transforms, we quickly determine whether these transforms will bring potential performance increase in a video coding system.