Multi-scale structural image quality assessment based on two-stage low-level features☆
Introduction
In recent years, there has been an increasing interest in developing objective image quality assessment (IQA) method that can automatically predict human behaviors in evaluating image quality. Such perceptual IQA measures play an important role in a broad range of applications such as image transmission, compression, enhancement, and super-resolution. Research on objective IQA aims to use computational models to measure the image quality consistently with human subjective evaluation. In this paper, the discussion is confined to full-reference (FR) methods.
In the early stage, the most widely used FR IQA are peak signal–to-noise ratio (PSNR) and mean squared error (MSE), which compute the intensity difference of images and they do not correlate well with perceived quality [1]. Some later developed models attempt to simulate the functionality of human visual system (HVS), which include noise quality measure (NQM) [2], visual signal-to-noise ratio (VSNR) [3], and information fidelity criterion (IFC) [4]. However, the HVS is a complex and highly non-linear system. So these approaches must rely on a number of strong assumptions and generalizations. The single-scale similarity index (SSIM) proposed in [5] can be considered as a milestone of the development of IQA metrics, which is based on the hypothesis that HVS is highly sensitive to the structural information in the image. The multi-scale extension of SSIM (MS-SSIM) provides better result than SSIM [6]. Recently, Lin Zhang et al. proposed an IQA metric named FSIM (feature similarity index metric) [7], which demonstrates much higher consistence with subjective evaluation than the other IQA metrics. However, the similarity index method introduced in Ref. [7] is a single-scale approach. This is considered to be a drawback of the method because the right scale depends on viewing conditions (e.g. display resolution and viewing distance), so it is not very well matched to the perceived visual quality.
To address the above-mentioned issues, this paper proposed a multi-scale structural similarity method based on two-stage low-level features, named MS-FSIM. The contribution of this work is that it is unique to extend the FSIM from single-scale to multi-scale, and it weights the different two-stage feature importance on each scale to achieve a state-of-the-art image quality evaluation between the objective metric and the subjective value. Efficiency of MS-FSIM and more quantitative results are corroborated by the extensive experimental results.
The remainder of this paper is organized as follows. Section 2 describes the single-scale feature similarity index (FSIM), which includes the computation of phase congruency (PC) and gradient magnitude (GM). Section 3 presents in detail the proposed multi-scale structural image quality assessment based on two-stage low-level features (MS-FSIM). Section 4 gives the experimental results and related discussions. Finally, Section 5 concludes the paper.
Section snippets
Phase congruency
Previous studies of psychophysics have revealed that visually discernible image features coincide with those points where Fourier waves at different frequencies have different phase congruency (PC) [8], it has been exploited as features by some biometrics researchers for recognition [9], [10], [11].
As Ref. [8] mentioned earlier, the human visual system has the capacity to simulate convolution by odd and even symmetric filters in quadrature. With respect to the quadrature pair of filters, the
Multi-scale feature similarity
As it is well-known, the perceivability of image details varies with a number of different factors, such as the sampling density of the image, the distance from the image plane to the observer, and the perceptual sensitivity of different observers. For this reason, the single-scale method described in Section 2.3 is appropriate only for specific cases in practice.
Multi-scale method is a convenient way to incorporate above different factors and reflect image details adequately. In this paper, we
Experimental results and discussion
In the publicly available IQA databases, we select five image databases including TID2008, LIVE, IVC, MICT and A57 for the proposed algorithm validation and comparison. The performance of the proposed MS-FSIM index will be evaluated and compared with other seven representative IQA metrics, including some state-of-the-art metrics, such as SSIM, MS-SSIM, FSIM, and IFC. For evaluation performance comparison, four commonly used metrics are used to reflect the comparable result, which including
Conclusion
In this paper, a novel IQA model, namely multi-scale feature similarity index metric (MS-FSIM), was proposed. It is based on the belief that the image degradation was hided in multi-scale low-level features, which represent abundant details of the image visual quality faithfully. It provides more flexibility than single-scale approach in incorporating the variations of image resolution and viewing conditions. Experiments show that when appropriate parameters are set, the multi-scale framework
Acknowledgements
The work is supported by Science Research Program of Hubei provincial Science &Technology Department of China (No. 2012FFC02601), Science Research Program of Hubei Provincial Department of Education of China (No. Q20111907), Science Research Program of Sichuan Provincial Department of Education of China (No. 11ZB073), Science Research Program of Visual Computing and Virtual Reality Key Laboratory of Sichuan Province (No. PJ201113), and the National Natural Science Foundation of China (No.
Li Guo received her MS degrees from Huazhong University of Science and Technology, China, in 2004 and received her Ph.D. degree in communication and information system from Sichuan University, China, in 2013. Her current research interests are image processing, information display, video super-resolution reconstruction and computer vision.
References (16)
Image quality metrics: PSNR vs. SSIM
ICPR
(2010)- et al.
Image quality assessment based on a degradation model
IEEE Trans Image Process
(2000) - et al.
VSNR: a wavelet-based visual signal-to-noise ratio for natural images
IEEE Trans Image Process
(2007) - et al.
An information fidelity criterion for image quality assessment using natural scene statistics
IEEE Trans Image Process
(2005) - et al.
Image quality assessment: from error visibility to structural similarity
IEEE Trans Image Process
(2004) - et al.
Multi-scale structural similarity for image quality assessment
IEEE Asilomar Conf Signal Syst Comput
(2003) - et al.
FSIM: a feature similarity index for image quality assessment
IEEE Trans Image Process
(2011) - et al.
A novel neighborhood defined feature selection on phase congruency images for recognition of faces with extreme variations
Int J Technol
(2007)
Cited by (0)
Li Guo received her MS degrees from Huazhong University of Science and Technology, China, in 2004 and received her Ph.D. degree in communication and information system from Sichuan University, China, in 2013. Her current research interests are image processing, information display, video super-resolution reconstruction and computer vision.
Wei-long Chen received his BS and MS degrees from Sichuan Normal University, China, in 2004 and 2007 respectively and received his Ph.D. degree in communication and information system from Sichuan University, China, in 2011. His current research interests are image processing and video super-resolution reconstruction.
Yu Liao is now a Ph.D. student of Sichuan University, China. His interest is laser communication and optical material.
Hong-hua Liao received his MS degree from Chongqing University, China, in 2006 and the Ph.D. degree from Huazhong University of Science and Technology, China, in 2010. His work focuses on the digital and analogue signal processing, the programmable devices’ application, and the design of micro-total analysis system and its application.
Jia Hu received his Ph.D. degree in Computing from the University of Bradford in 2010, and the MS degree in physical electronics and communication engineering from Huazhong University of Science and Technology, China, in 2006. His research interests include wireless networking, mobile computing, cross-layer optimization, and resource management.
- ☆
Reviews processed and approved for publication by Editor-in-Chief Dr. Manu Malek.