Correlation-based approach to color image compression

Abstract

Most coding techniques for color image compression employ a de-correlation approach—the RGB primaries are transformed into a de-correlated color space, such as YUV or YCbCr, then the de-correlated color components are encoded separately. Examples of this approach are the JPEG and JPEG2000 image compression standards. A different method, of a correlation-based approach (CBA), is presented in this paper. Instead of de-correlating the color primaries, we employ the existing inter-color correlation to approximate two of the components as a parametric function of the third one, called the base component. We then propose to encode the parameters of the approximation function and part of the approximation errors. We use the DCT (discrete cosine transform) block transform to enhance the algorithm's performance. Thus the approximation of two of the color components based on the third color is performed for each DCT subband separately. We use the rate-distortion theory of subband transform coders to optimize the algorithm's bits allocation for each subband and to find the optimal color components transform to be applied prior to coding. This pre-processing stage is similar to the use of the RGB to YUV transform in JPEG and may further enhance the algorithm's performance. We introduce and compare two versions of the new algorithm and show that by using a Laplacian probability model for the DCT coefficients as well as down-sampling the subordinate colors, the compression results are further improved. Simulation results are provided showing that the new CBA algorithms are superior to presently available algorithms based on the common de-correlation approach, such as JPEG.

Introduction

Recently a new algorithm for color image compression was introduced by Goffman and Porat [9]. This algorithm utilizes the high inter-color correlations between RGB color components of natural images [2], [9], [11], [14], [18], [22] without transforming them into another color domain. It does so by dividing the image into square blocks, expanding two of the color primaries (the dependent colors) as a polynomial function of the third (base) color for each block. This way only the polynomial coefficients are encoded for each block of the dependent colors, whereas the base color component is encoded by any monochromatic compression technique. The choice of the base was examined in [9] with general tendency to choose the Green. The conclusion in [9] was that the new algorithm produces less color artifacts than the common de-correlation approach at high compression ratios. The de-correlation approach (such as JPEG [20] and JPEG2000 [10], [16]) consists of transforming the color components into a de-correlated color space, then separately encoding each of the obtained color components. This approach is the most common for color images; however, it is not necessarily the optimal one. Additional examples of this approach can be found in [12], [21].

The algorithm by Goffman and Porat may be considered as an example of a correlation-based approach (CBA) to color image compression. In this work we introduce two new algorithms based on this approach. We use the DCT (discrete cosine transform) block transform [17] to enhance the compression performance. Thus the expansion of the dependent colors is done for each DCT subband instead of for each image block. However, not only the expansion coefficients are coded, but also the approximation errors for part of the subbands. Since the DCT block transform is a special case of a subband transform, we propose to employ the recently developed rate-distortion (R-D) theory for subband transform coders [6]. This theory allows us not only to find the optimal bits allocation for the subbands in MSE (mean square error) sense, but also the optimal color components transform (CCT) as an efficient pre-processing stage.

The structure of this paper is as follows. In Sections 1.1, 1.2, and 1.3 we review the R-D theory for subband transform coders and the algorithms for calculating the optimal subband rates and the corresponding PCM quantization steps. In Sections 2 and 3 we introduce two versions of the new algorithm and discuss how the R-D theory is used in their optimization. Section 4 deals with optimizing the CCT. Then in Section 5 down-sampling (DS) of the subordinate colors and use of the Laplacian probability model for the DCT coefficients are described. This model allows reduction of the algorithms’ complexity and provides lower coding distortion for the same transmission rate. Another potential improvement of the algorithms’ performance is presented in Section 6. Simulation results of the new algorithms and their comparison to JPEG as a representative of the de-correlation approach are discussed in Section 7. Finally, conclusions and summary are given in Section 8.

In [6] a general subband transform coder for color images was considered, based on the following steps.

• Pre-processing: apply a CCT to the RGB color components of the given image. Denoting the RGB components in vector form as $\mathbf{x}=[RGB{]}^{\mathrm{T}}$ and the new color components as $\tilde{\mathbf{x}}=[C1C2C3{]}^{\mathrm{T}}$, this stage can be written as

  $$\tilde{\mathbf{x}}=\mathbf{Mx}$$

  for some $3\times 3$ CCT matrix M.

• Apply a subband transform, such as DCT or DWT (discrete wavelet tree) or any filter bank decomposition to each color component. The subband transform is usually assumed to be non-expansive, i.e., it transforms a signal of length N to a signal with the same length.

• Quantize the coefficients of each subband of each color component. A uniform scalar quantizer was considered. We refer to this stage as applying the PCM (pulse code modulation) scheme.

• Post-processing: encode the quantized coefficients in a lossless manner, such as run-length coding, delta modulation or entropy coding.

At high rates R the R-D behavior of the basic PCM scheme applied to a random signal x with variance ${\sigma }_x^2$ is [7], [19]

$$d(R)={\epsilon }^2{\sigma }_x^2{2}^{-2R}\text{,}$$

where ${\epsilon }^2$ is a constant dependent upon the distribution of x. This is only an approximate behavior or model; however, based on (2) the R-D model of a general monochromatic subband coder with B subbands can be expressed as

$$d^{\mathit{SC}}(\{R_b\})=\sum _{b=0}^{B-1}{\eta }_b{G}_b{d}_b(R_b)=\sum _{b=0}^{B-1}{\eta }_b{G}_b{\sigma }_b^2{\epsilon }^2{2}^{-2R_b}\text{.}$$

Here $d_b(R_b)$ is the MSE of subband b $(b\in 0,1,\dots ,B-1)$, ${\sigma }_b^2$ is its variance, $G_b$ is its energy gain [19], $R_b$ is the rate allocated to it, and ${\eta }_b$ is its sample rate. The sample rate of a subband is equal to the relative part of the number of coefficients in it from the total number of samples in the subband transformed signal. For a uniform subband transform, such as the block DCT, ${\eta }_b=1/B$.

Consider now a color image. The coding algorithm described in the beginning of this section may be regarded as applying a CCT to the image, followed by monochromatic subband coding of each of the new color components. The R-D model of this algorithm is

$$d(\{R_{\mathit{bi}}\},\mathbf{M})=\frac{1}{3}\sum _{i=1}^3\sum _{b=0}^{B-1}{\eta }_b{G}_b{\sigma }_{\mathit{bi}}^2{\epsilon }_i^2{e}^{-{\mathit{aR}}_{\mathit{bi}}}(({\mathbf{MM}}^{\mathrm{T}}{)}^{-1}{)}_{\mathit{ii}}\text{,}$$

where $a=2\ln 2$ and ${\sigma }_{\mathit{bi}}^2$ and $R_{\mathit{bi}}$ remain the same, but for subband b of color component i $(i\in 1,2,3)$. Note that standard entropy coding is assumed here in the post-processing stage or alternatively this stage is not taken into account.

Minimizing the expression of Eq. (4) under the rate constraint

$$\sum _{i=1}^3\sum _{b=0}^{B-1}{\eta }_b{R}_{\mathit{bi}}=R$$

as well as non-negativity constraints for the rates lead to the optimal subband rates allocation. If we denote the set of non-zero (or active) rates in the color component i by ${\mathrm{Act}}_i$, i.e., ${\mathrm{Act}}_i\triangleq \{b\in [0,B-1]|R_{\mathit{bi}}>0\}$, then the active optimal rates are given by

$$R_{\mathit{bi}}=\frac{R}{{\sum }_{j=1}^3{\xi }_j}+\frac{1}{a}\ln \left(\frac{{\epsilon }_i^2{G}_b{\sigma }_{\mathit{bi}}^2(({\mathbf{MM}}^{\mathrm{T}}{)}^{-1}{)}_{\mathit{ii}}}{{\prod }_{k=1}^3[({\epsilon }_k^2{\mathit{GMA}}_k({\mathbf{MM}}^{\mathrm{T}}{)}^{-1}{)}_{\mathit{kk}}{]}^{{\xi }_k/{\sum }_{j=1}^3{\xi }_i}}\right)\text{,}$$

where

$${\xi }_i\triangleq \sum _{b\in {\mathrm{Act}}_i}{\eta }_b,{\mathit{GMA}}_i\triangleq \prod _{b\in {\mathrm{Act}}_i}(G_b{\sigma }_{\mathit{bi}}^2{)}^{{\eta }_b/{\xi }_i}\text{.}$$

The rate constraint of (5) and the optimal rates of (6) do not account for DS of some of the color components as, for example, is done in JPEG [20]. To do so a DS factor ${\alpha }_i$ is introduced for color component i, so that if the DS is by a factor of 2 horizontally and vertically then

Thus the rate constraint becomes

$$\sum _{i=1}^3{\alpha }_i\sum _{b=0}^{B-1}{\eta }_b{R}_{\mathit{bi}}=R\text{,}$$

and the solution for the rates is

$$R_{\mathit{bi}}=\frac{R}{{\sum }_{j=1}^3{\alpha }_j{\xi }_j}+\frac{1}{a}\ln \phantom{R_{\mathit{bi}}=}\left(\frac{\frac{{\epsilon }_i^2{G}_b{\sigma }_{\mathit{bi}}^2(({\mathbf{MM}}^{\mathrm{T}}{)}^{-1}{)}_{\mathit{ii}}}{{\alpha }_i}}{{\prod }_{k=1}^3{\left(\frac{(({\mathbf{MM}}^{\mathrm{T}}{)}^{-1}{)}_{\mathit{kk}}{\epsilon }_k^2{\mathit{GMA}}_k}{{\alpha }_k}\right)}^{{\alpha }_k{\xi }_k/{\sum }_{j=1}^3{\alpha }_j{\xi }_j}}\right)\text{,}(b\in {\mathrm{Act}}_i)\text{.}$$

There is still a question of how the active rates are determined. The iterative algorithm introduced in [6] for this purpose is described in the next subsection.

The following algorithm can be used to find the active subbands iteratively:

• Assume all the subbands are active and calculate the rates.

• While some $R_{\mathit{bi}}<0$:

  • Set ${\mathrm{Act}}_i=\{b\in [0,B-1]|R_{\mathit{bi}}>0\}$.

  • Calculate new rates.

• Check that the Lagrange multipliers ${\mu }_{\mathit{bi}}⩾0$, where

  $${\mu }_{\mathit{bi}}=\left\{\begin{array}{cc}\frac{a}{3}{\eta }_b(({\mathbf{MM}}^{\mathrm{T}}{)}^{-1}{)}_{\mathit{ii}}{\epsilon }_i^2\hfill & b\notin {\mathrm{Act}}_i,\hfill \\ (G_0{\sigma }_{0i}^2{e}^{-{\mathit{aR}}_{0i}}-G_b{\sigma }_{\mathit{bi}}^2),\hfill & \hfill \\ 0,\hfill & b\in {\mathrm{Act}}_i.\hfill \end{array}\right.$$

Here ${\sigma }_{0i}^2$ and $R_{0i}$ are the variance and rate, respectively, of subband 0 of component i. This subband can be any subband that can be assumed to be active always. Usually we would choose the subband with the maximal energy (variance) for that component, e.g., the DC subband for the DCT.

Once the optimal rates have been determined, there is still the question of the size of the PCM quantization steps to choose to achieve these rates. The quantization steps algorithm proposed in [3] is given next.

The algorithm consists of the following stages:

• Calculate the optimal rates $R_{\mathit{bi}}^{*}$ (using (6) or (9)).

• Set some initial quantization steps ${\Delta }_{\mathit{bi}}$ and calculate the resulting rates $R_{\mathit{bi}}$. The rate $R_{\mathit{bi}}$ is the entropy of subband b of color component i.

• Update the quantization steps according to

  $${\Delta }_{\mathit{bi}}^{\mathrm{new}}={\Delta }_{\mathit{bi}}{2}^{-(R_{\mathit{bi}}^{*}-R_{\mathit{bi}})}$$

  until the optimal rates $R_{\mathit{bi}}^{*}$ are sufficiently close, i.e., $E(|R_{\mathit{bi}}^{*}-R_{\mathit{bi}}|)<\epsilon$ for some small constant $\epsilon$. $E()$ stands here for statistical mean.

This algorithm provides a means of how to deal with the active subbands. The coefficients of the non-active subbands are zeroed.