A reversibility-gain model for integer Karhunen-Loève transform design in video coding

Abstract

Karhunen-Loève transform (KLT) is the optimal transform that minimizes distortion at a given bit allocation for Gaussian source. As a KLT matrix usually contains non-integers, integer-KLT design is a classical problem. In this paper, a joint reversibility-gain (R-G) model is proposed for integer-KLT design in video coding. Specifically, the ‘reversibility’ is modeled according to distortion analysis in using forward and inverse integer transform without quantization. It not only measures how invertible a transform is, but also bounds the distortion introduced by the non-orthonormal integer transform process. The ‘gain’ means transform coding gain (TCG), which is a widely used criterion for transform design in video coding. Since KLT maximizes the TCG under some assumptions, here we define the TCG loss ratio (LR) to measure how much coding gain an integer-KLT loses when compared with the original KLT. Thus, the R-G model can be explained as follows: subject to a certain TCG LR, an integer- KLT with the best reversibility is the optimal integer transform for a given non-integer-KLT. Experimental results show that the R-G model can guide the design of integer-KLTs with good performance.