Biologically inspired image quality assessment

Highlights

• We propose a novel IQA approach named biologically inspired feature similarity (BIFS), which is demonstrated to be highly consistent with the human perception.

• In the proposed approach, biologically inspired features (BIFs) of the test image and the relevant reference image are first extracted.

• Afterwards, local similarities between the reference BIFs and the distorted ones are calculated and then combined to obtain a final quality index.

• Thorough experiments on a number of IQA databases demonstrate that the proposed method is highly effective and robust, and outperform state-of-the-art FR-IQA methods across various datasets.

Abstract

Image quality assessment (IQA) aims at developing computational models that can precisely and automatically estimate human perceived image quality. To date, various IQA methods have been proposed to mimic the processing of the human visual system, with limited success. Here, we present a novel IQA approach named biologically inspired feature similarity (BIFS), which is demonstrated to be highly consistent with the human perception. In the proposed approach, biologically inspired features (BIFs) of the test image and the relevant reference image are first extracted. Afterwards, local similarities between the reference BIFs and the distorted ones are calculated and then combined to obtain a final quality index. Thorough experiments on a number of IQA databases demonstrate that the proposed method is highly effective and robust, and outperform state-of-the-art FR-IQA methods across various datasets.

Introduction

The past decades have witnessed a dramatic increase in the number of images with the tremendous development of social networking websites, smartphones, and cameras. And various systems have been developed to deal with such a large scale of images. In these systems, image quality usually plays a significant role. For example, images of poor quality may lead to obstacles in learning or applying such systems for practical applications, e.g. scene recognition [1], image retrieval [2], and so on. In addition, image quality can be adopted as a criterion for evaluating the performance of image processing systems [3], [4], [5], optimizing image processing algorithms, and monitoring the working condition of devices [6]. Thus it is meaningful to develop image quality assessment (IQA) methods that can precisely and automatically estimate human perceived image quality.

In recent years, many IQA methods have been developed and we can classify them into three classes [6]: full-reference (FR) IQA [7], reduced-reference (RR) IQA [8], [9], and no-reference (NR) or blind IQA [10], [11]. FR-IQA methods need all the information of the reference image, i.e. the undistorted version of the test image, is needed. In contrast, RR-IQA and NR-IQA methods only need part of or none of the information about the reference image. Consequently, the quality prediction accuracies of FR-IQA methods are usually better than present RR-IQA and NR-IQA methods.

Generally speaking, the intrinsic idea of FR-IQA is to estimate the quality of a test image by measuring the similarity or difference between the test image and the corresponding reference image. For example, in peak signal-to-noise ratio (PSNR) and root mean squared error (RMSE), the most widely used two IQA methods, the differences between the reference image and the test image are calculated pixel by pixel, and then combined into a single value. Because PSNR and MSE do not always consistent well with human perception [12], great efforts have been paid to develop progressive methods for quality assessment in the past decades [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28]. And many of them have shown impressive and inspiring consistency with human perception over a large range of datasets [26], [27], [28].

Since the goal of IQA is to approximate human beings’ judgments of image quality, it is meaningful to develop IQA methods that mimic the perception mechanism of the human visual system (HVS). Although many attempts have been proposed, most of them only consider some particular properties of HVS, e.g. contrast sensitivity function (CSF) [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], just noticeable difference (JND) [30], and visual attention (VA) [31], etc. Usually they do not perform as well as state-of-the-art FR-IQA methods. To date, only limited IQA methods have been proposed to formula the processing in the visual cortex [27]. And the properties of primary visual cortex, V1, have not been well explored for IQA, although neuroscientists have demonstrated that V1 plays a much significant in visual processing [32].

In this paper, we utilize biologically inspired feature (BIF) models [33] to mimic the properties of (S1) and complex (C1) cells in V1, and construct a novel IQA index by measuring the similarity between the BIFs of the reference image and those of the test image. Although BIFs have been introduced to FR-IQA before, it was adopted for estimating visual attention [34]. In contrast, in the proposed method, BIFs are deplored for representing the input image in the primary visual cortex and directly utilized for quality prediction. Thorough experiments conducted on various IQA databases demonstrate that the proposed method is in highly consistency with human perception and outperform state-of-the-art FR-IQA methods across a number of datasets. The highlights of the proposed method are summarized below as follows:

• We explore BIF for FR-IQA by employing it to mimic the processing in the primary visual cortex;

• We construct a novel FR-IQA framework by measuring the similarity between the BIFs of the test image and the BIFs of the corresponding reference image; and

• Thorough experiments on existing databases demonstrate that the proposed method is highly comparable with state-of-the-art FR-IQA methods.

• The rest of the paper is organized as follows. Section II introduces the calculation of BIFs. In Section III, we present the framework of the proposed quality evaluation method. Extensive experiments conducted on standard IQA datasets are presented and analyzed in Section IV. Section V concludes the paper.