Image quality assessment using a novel region smoothness measure

Highlights

• Abstracting the idea of three-component image structural model into the higher solitary concept.

• An efficient FR IQA method is presented based on the region smoothness fidelity.

• The proposed IQA method employs our recently suggested MSER-based smoothness measure.

Abstract

One of the most efficient descriptions of image structure, which has been widely used in image quality assessment (IQA) studies, is the three-components model. Based on this model, the major structural components of an image are edges, textures and flat regions. We found that this model is basically derived from the abstract concept of image region smoothness. Indeed, each of these three components, is a particular region with special smoothness characteristics. Inspired by this fact, we developed an efficient general-purpose full-reference IQA technique, in which the amount of region smoothness degradation is gauged using our efficient MSER (maximally stable extremal region)-based region smoothness measure. For this, we build a block-based smoothness similarity map, and extract the image quality score, using a percentile averaging scheme. Experimental results are provided on popular benchmark databases, which confirm that the proposed approach has a reasonable prediction performance compared to the state-of-the-art image quality metrics.

Introduction

Image quality assessment (IQA) is one of the most fundamental yet challenging problems in image processing and machine vision studies, because digital images are subject to a wide variety of distortion during acquisition, storage, processing, transmission, compression, watermarking, fusing, and even enhancement. Intuitively human being is the final and sometimes the best judge, and his/her opinion, which is often referred to as the subjective quality score, determines the quality of image. But the subjective assessment is very time-consuming and inefficient, and cannot be used for real-time applications. Considerable efforts have been performed to develop objective quality metrics to automatically predict the human judges, accurately, and provide scores as close as possible to the subjective ones [1].

In terms of dependence to the reference image, the existing objective IQA measures can be classified into three major categories, i.e., Full-reference (FR), Reduced-Reference (RR) and No-Reference (NR) ones. FR models require the original pristine image known as the reference image. These models are widely used to evaluate the performance of image processing algorithms and are frequently employed in optimization procedures of image enhancement methods [2], [3]. Reduced-reference IQA methods, which only need the quality aware features from reference images, are useful to assess the quality of received images in a noisy transformation channel [4], [5]. The third class is No-reference or blind IQA methods, which do not require any information from reference images [6], [7]. The NR methods are more challenging due to their reference image independency.

One of the most suitable paradigm for IQA measure design, is modeling the human visual system (HVS). Based on the HVS properties understood so far, the main concern in HVS is the image structure. Indeed, the HVS tries to extract the image structure for understanding the image contents and subsequently, finding out the image quality. The main difficulty arises here is the precise definition of image structure. One interesting approach supported in [8], [9], [10], [11], is a three-components image model, in which an image is decomposed into edges, textures and smooth regions, as its major structural components, and then different quality evaluators are employed for each.

In our research, we noticed that although these different image components have different importance for vision perception, the main cause of these differences lies on the image region smoothness, as a higher abstract concept. Indeed, the main difference between an edge block and a texture block is the difference in their smoothness characteristics. Similarly, a texture block and a flat block, have different smoothness nature. Hence, by tracking and gauging the image smoothness changes, one can efficiently evaluate the amount of image distortions.

The major contribution of our work is the new definition of image structural information based on the region smoothness. We abstracted the idea of three-components image structural model (i.e. edge, texture and smooth regions), into the higher solitary concept, i.e., image region smoothness. By this, we can track the image structural changes just by considering the image smoothness modifications instead of tracking the changes in three different components.

To assess the image region smoothness, we need a proper smoothness evaluator. There have been several studies in the literature introducing image smoothness descriptors, including variance-based smoothness measure [12], Gradient-based methods [13], [14], quadratic variation based approaches [15], and Laplacian-related smoothness evaluators [15]. Here we develop a novel image smoothness descriptor based on the Maximally Stable Extremal Regions (MSER) concept, which has been recently introduced in our previous work [16], to propose an efficient FR image quality assessment method, gauging the amount of image degradation, for a wide variety of image distortions. This paper introduces a new measure, dubbed as MSER smoothness-based quality index (MSER-SQI), which is merely based on the image smoothness concept. Our experiments on common used image databases show the outstanding functionality of the proposed method.

The rest of this paper is organized as follows. In Section 2, we introduce the MSER concepts and briefly describe our recently proposed MSER-based block smoothness measurement approach. In addition, we compare the ability of distortion severity discrimination of the proposed smoothness measure, with some other existing smoothness measures, in this section. In Section 3, the proposed MSER smoothness-based image quality score is described. The implementation protocols and experimental results are presented in Section 4. Finally, we conclude the manuscript in Section 5.