Image quality assessment by discrete orthogonal moments

Abstract

This paper proposes a novel full-reference quality assessment (QA) metric that automatically assesses the quality of an image in the discrete orthogonal moments domain. This metric is constructed by representing the spatial information of an image using low order moments. The computation, up to fourth order moments, is performed on each individual $(8\times 8)$ non-overlapping block for both the test and reference images. Then, the computed moments of both the test and reference images are combined in order to determine the moment correlation index of each block in each order. The number of moment correlation indices used in this study is nine. Next, the mean of each moment correlation index is computed and thereafter the single quality interpretation of the test image with respect to its reference is determined by taking the mean value of the computed means of all the moment correlation indices. The proposed objective metrics based on two discrete orthogonal moments, Tchebichef and Krawtchouk moments, are developed and their performances are evaluated by comparing them with subjective ratings on several publicly available databases. The proposed discrete orthogonal moments based metric performs competitively well with the state-of-the-art models in terms of quality prediction while outperforms them in terms of computational speed.

Introduction

Advancement made in digital imaging and communication techniques over the past two decades have led to an abundance of new applications for human end-use. At the same time, demand for better quality has also grown in tandem. Measuring and interpreting the quality of acquired images against a database of known, but diverse reference images has become one of the most challenging tasks in visual form analysis. Given the situation, probably the best way to evaluate the quality of an image is to seek the opinions of human observers as they are representative of the ultimate receivers in most image processing environment. Towards this end, the subjective metric, mean opinion scores (MOS) based on the characteristics of the human visual system (HVS), has been developed to measure perceptual quality [1]. Another subjective metric with similar characteristics is the difference mean opinion scores (DMOS) [2]. Nevertheless, based on the feedback from human observers, subjective evaluation is not possible to be incorporated into automatic systems because of their inability to render observations in real time.

As a result of aforementioned limitation, a different approach which evaluates the image quality objectively that is consistent with the subjective human evaluation has become the major research direction in the image/video quality assessment (QA). It has been proven to be invaluable in many real world applications particularly in the telecommunication industry, multimedia and computer vision operations.

The objective evaluation involves designing a mathematically defined metric that is low in computational complexity, and independent of viewing conditions and human observers. The standard approach for this assessment is by using a full-reference (FR) QA, in which the image similarity or fidelity is measured against a ‘reference’ image/video of ‘perfect quality’ (no loss in fidelity with the original scene). Based on this concept, an abundance of objective metrics have been developed but differ in approaches. In spite of this, they deliver a single consistent score that generalizes the quality of an image or video sequence in various viewing conditions [3], [4], [5], [6], [7], [8], [9], [10].

Of these, the most widely used FR objective metrics are mean squared error (MSE) and its variants such as root mean squared error (RMSE), signal-to-noise ratio (SNR) and peak signal-to-noise ratio (PSNR). The metrics are based on the statistical error sensitivity approach and Minkowski error pooling concept that make them computationally attractive. Nevertheless, they do not correlate well with perceived quality measurement because human perception of image/video distortions and artifacts is not taken into consideration. It has been justified that by integrating with some HVS characteristics, the performance of MSE or its variants can be improved upon [11], [12], [13].

It has also been reported that none of the other objective image quality metrics thus far has shown any clear advantage over simple mathematical measures such as PSNR under strict testing conditions and different image distortion environment [2], [14], [15]. Therefore, researchers are currently focused on improving PSNR or introducing more efficient assessment algorithms.

Orthogonal moments have been widely used in various areas in image processing for the past few decades. Their applications include pattern or object classification [16], [17], [18], [19], [20], [21], [22], text and characters recognition [23], [24], classification of multi-spectral texture [25], texture retrieval [26], image registration [27], image segmentation and edge detection [28], [29], [30], image compression [31], face recognition [32], [33], stereo vision [34], palmprint verification [35], content-based image retrieval [36], pose estimation of three-dimensional objects [37], focus measure [38], image segmentation watermarking [39], [40], etc. The successful implementation of the orthogonal moments in the aforementioned applications is mainly due to their inherent properties and some manipulations on the computed moment values. Their inherent properties include invariance properties, information compactness, oscillating kernels, conveying spatial and phase information of an image.

Orthogonal moments are good signal descriptors with their low order components¹ are sufficient to provide discriminant power in pattern or object recognition [41], [42]. This low order moments set provides a global, general but coarse description about the object. With the moment order increases, more and more fine information about the object is included in the computed moments. This is due to the inherent characteristics of orthogonal polynomials which oscillate like sine and cosine functions. The oscillating functions sample the image pixels of different location with different sampling rate when different moment orders are used. For the case of discrete orthogonal moments such as Tchebichef and Krawtchouk moments, image reconstruction can be performed without any loss if up to the maximum order² of the discrete orthogonal moments are used.

One of the important applications of orthogonal moments in image process is the edge detection of the objects inside an image. This is accomplished through the ability of orthogonal polynomials to function as a filters bank of different spatial frequencies, by changing the order of sine and cosine-like polynomials. Edge information of the objects (such as in the directions of horizontal, vertical and diagonal) is able to be extracted when this filters bank is moving across the whole image. Different combinations of orthogonal polynomials extract the edge information of different directions. The phenomena can obviously observed through the orthogonal moments basis functions image.

Inspired by their successful applications in various areas of image processing, a novel metric for quantifying the quality of images objectively based on the correlation indices defined in the discrete orthogonal moment domain is proposed in this paper. It is believed that the orthogonal moments can be applied as a features set in measuring how an image is departure from its reference due to their favorable properties such as information compactness in representation, spatial globality, spatial-frequency filters bank, image spatial information representation. These properties facilitate recognition through global structures corresponding to spatial frequency and spatial localization. The reader can refer to [43] for the details of the essential characteristics and computation properties of various moment functions and their roles in image analysis. Nevertheless, one of the main drawbacks of orthogonal moments is their computational complexity. This problem is usually handled by simplifying their computation via recurrence relationships and symmetrical property between successive polynomials [44], [45], [46], [47], [48].

The organization of this paper is as follows. A brief background of the state-of-the-art FR QA models based on error sensitivity, structural similarity, natural scene statistics, and human vision system (HVS) is given in Section 2. Brief information on the characteristics of discrete orthogonal moments and their correlation relationships with image contents are given in Section 3. The mathematical formulation of the proposed discrete orthogonal moments-based models is presented in Section 4. The performance of the proposed model is validated through experiments using natural images of various distortions as well as comparative analysis with other models as shown in Section 5. The study is concluded in Section 6.