No-reference image quality assessment in contourlet domain
Introduction
Among most of the applications, the visual information in an image is ultimately received by human beings, it is intuitively reasonable to score the image quality subjectively [1]. However, subjective image quality assessment (IQA) methods are expensive and time consuming and so they cannot be easily and routinely performed for many scenarios, e.g., real time systems. The current alternative to subjective IQA is objective IQA [3], which tries to assign scores that are in meaningful agreement with subjective result. These results can be used to automatically assess the image quality.
Objective quality metrics can be divided into reference [6], [8], [26], [27], [28], [29] and no-reference [9] methods, depending on whether the method utilizes reference information. Reference methods usually provide more precise assessments of image quality than non-reference methods. The objective measures used in reference methods include peak signal-to-noise ratio (PSNR), structural similarity (SSIM) [10] and the visual information fidelity (VIF) [11]. These methods are found to be highly correlated with subjective scores. However, their application is limited because the reference information may not be available or too expensive to be obtained in many scenarios [2], [4].
NR methods output evaluation results without the reference information. Instead, they rely on strong hypotheses of one or a set of predefined distortions, e.g., the blocking effect in JPEG, the ringing and blurring artifacts in JPEG2000. Wang et al. [12] proposed an effective and efficient quality assessment model for JPEG images. Bovik and Liu [13] modeled the blocking-artifacts to measure the compressed image quality. Mei et al. [14] proposed a novel spatio-temporal quality assessment scheme by using low-level content features for home videos. Luo and Tang [15] evaluated visual quality by using high-level semantic features are utilized. Badu et al. [16] applied the edge amplitude and length, the background luminance, and activity to IQA. Sheikh et al. [9] proposed an NR IQA metric that uses natural scene statistics (NSS) for JPEG2000 compressed images.
The existing state-of-art approach employs the NSS model [9] to do IQA. This is based on the observation that natural images exhibit certain common statistical characteristics which can be represented by a mathematical model and disturbed by the distorted processing. For IQA, degree of distortion can be quantized by measuring the change of statistical characteristics model. Sheikh's NSS model metrics models the marginal distribution [17] and the joint statistics [18], [19] of wavelet coefficients to evaluate JPEG2000 compressed natural images, which outperforms many other existing methods. However, the experimental results also show that this approach is only applicable to JPEG2000 compressed images. In other words, the method is only effective for the image which is processed based on wavelet transform. Moreover, wavelet transforms fail to explicitly capture directional information which is necessary for image modeling. Therefore, there is still a big space to further improve the performance of NR IQA [5].
In this paper we propose to deal with the aforementioned problems by applying contourlets. The principle advantage of applying contourlet transform is that its characteristics can be used to model images that suffer from directional multiscale dependencies, which are sensitive to image degradation and therefore can be used to evaluate the quality without the corresponding reference images. The mutual information between contourlet coefficients at different spaces, scales and directions are first modeled by calculating the joint histogram of reference and predicted coefficients. Then, in order to eliminate the effect of image content, an image-dependent threshold is employed to divide the joint space as two regions. Finally, image quality can be predicted by pooling and mapping a signification region, which is most represented for distortion, to suitable measurement. Because the contours and smooth curves are dominant in image structure which is most sensitive for human eyes, modeling these features using contourlet transform is an effectual metrics for IQA [7].
The remainder of the paper is organized as follows. Section 2 introduces NR IQA framework of employing the image modeling in contourlet domain. In Section 3, experimental results are given. Finally, conclusion is showed in Section 4.
Section snippets
Image modeling for no-reference image quality assessment in contourlet domain
Based on the statement above, we develop an improved image model in contourlet domain to do image quality assessment, which quantifies the variation of the nonlinear dependencies between contourlet coefficients to measure the image degradation. Fig. 1 shows the framework of the proposed and it works with following stages: (1) contourlets [20] is utilized to achieve an optimal approximation rate of piecewise smooth functions, (2) the relationship of contourlet coefficients is represented by the
Simulation details
We validate our algorithm on the laboratory for image & video engineering (LIVE) database [22]. This database contains 29 high-resolution 24 bits/pixel RGB color images as reference and corresponding five types of distorted images: 175 JPEG, 169 JPEG2000, 145 White Noisy (WN), 145 Gaussian blurred (Gblur) and 145 Fast-Fading (FF) Rayleigh channel noisy images at a range of quality levels (the sample images are shown in Fig. 7). The difference mean opinion scores (DMOS) of the image is provided
Conclusion
In this paper, an improved image model is proposed to do image quality assessment without any reference by capturing the image structural information. The image is decomposed by contourlet into multiscale and multidirectional subbands. The nonlinear dependencies are captured by the joint histograms of the reference and estimated contourlet coefficients. An image-dependent threshold is employed to eliminate the influence of content. The final objective quality score is calculated by the
Acknowledgements
We thank constructive suggestions from anonymous reviewers. This research was supported by National Science Foundation of China (60771068, 60702061, 60832005), the Open-End Fund of National Laboratory of Pattern Recognition in China and National Laboratory of Automatic Target Recognition, Shenzhen University, China, the Program for Changjiang Scholars and innovative Research Team in University of China (IRT0645), the Nanyang Technological University Nanyang SUG Grant (under project number
Wen Lu received his B.Eng. degree from the Xidian University in 2002, his M.Phil. degrees from the Xidian University in 2006, his Ph.D. degree in Pattern Recognition & Intelligent System at the Xidian University in 2009. Currently, he is a lecturer with the School of Electrical Engineering in the Xidian University.
His research interests include image & video understand, visual quality assessment, and computational vision. He has published a book and around 20 scientific papers including IEEE
References (29)
- et al.
Perceptual blur and ringing metrics: application to JPEG2000
Signal Processing: Image Communication
(2004) - Methodology for the subjective assessment of the quality of television pictures, Recommendation ITU-R Rec. BT....
- et al.
Effect of silhouette quality on hard problems in gait recognition
IEEE Transactions on Systems, Man, and Cybernetics—Part B: Cybernetics
(2005) - et al.
Modern Image Quality Assessment
(2006) - et al.
Fingerprint image mosaicking by recursive ridge mapping
IEEE Transactions on Systems, Man, and Cybernetics—Part B: Cybernetics
(2007) Emulating human visual perception for measuring difference in images using an SPN graph approach
IEEE Transactions on Systems, Man, and Cybernetics—Part B: Cybernetics
(2002)- et al.
A statistical evaluation of recent full reference image quality assessment algorithms
IEEE Transactions on Image Processing
(2006) - et al.
Human visual system-based image enhancement and logarithmic contrast measure
IEEE Transactions on Systems, Man, and Cybernetics—Part B: Cybernetics
(2008) - et al.
Quality-aware images
IEEE Transactions on Image Processing
(2006) - et al.
No-reference quality assessment using natural scene statistics: JPEG2000
IEEE Transactions on Image Processing
(2005)
Image quality assessment: from error visibility to structural similarity
IEEE Transactions on Image Processing
An information fidelity criterion for image quality assessment using natural scene statistics
IEEE Transactions on Image Processing
Cited by (83)
QL-IQA: Learning distance distribution from quality levels for blind image quality assessment
2022, Signal Processing: Image CommunicationDeep blind image quality assessment based on multiple instance regression
2021, NeurocomputingCitation Excerpt :Since the reference information is not available in many applications, the blind IQA algorithms are the most promising technique and attract more and more attention recently. The IQA algorithms can be categoried as traditional algorithm [2–12] and deep learning based algorithm [13–27]. Convolutional neural network (CNN) based IQA has developed rapidly in recent years because of its outstanding performance [15–27].
Blind image quality assessment in the contourlet domain
2021, Signal Processing: Image CommunicationNo-reference screen content image quality assessment based on multi-region features
2020, NeurocomputingCitation Excerpt :Therefore, image quality assessment (IQA) plays a very important role in image processing community. Most of the existing IQA algorithms were designed for the natural images [1–10]. Compared with the previous methods including MSE and peak signal-to-noise ratio (PSNR), these methods have made great progresses in the design of better IQA algorithms by considering Natural Scene Statistics (NSS) and various human visual system (HVS) characteristics, such as image structure [1,2], edge information [3,4], visual saliency [5], NSS-based features [6–8], information fidelity [9], biologically inspired features [10], and so on.
No-reference quality assessment of blurred frames
2018, Procedia Computer Science
Wen Lu received his B.Eng. degree from the Xidian University in 2002, his M.Phil. degrees from the Xidian University in 2006, his Ph.D. degree in Pattern Recognition & Intelligent System at the Xidian University in 2009. Currently, he is a lecturer with the School of Electrical Engineering in the Xidian University.
His research interests include image & video understand, visual quality assessment, and computational vision. He has published a book and around 20 scientific papers including IEEE TIP, TSMC, etc. E-mail: [email protected]
Kai Zeng received his B.S. degree in Electrical Engineering from the Xidian University in 2006, and his M.S. degrees from the Xidian University in 2009. He is currently pursuing his Ph.D. degree in University of Waterloo, Canada. His research interests include image & video quality assessment. E-mail: [email protected]
Dacheng Tao received the B.Eng. degree from the University of Science and Technology of China (USTC), the M.Phil degree from the Chinese University of Hong Kong (CUHK), and the Ph.D. degree from the University of London (Lon). Currently, he is a Nanyang Assistant Professor with the School of Computer Engineering in the Nanyang Technological University and a Visiting Research Fellow in Lon. He is a Visiting Professor in the Xi Dian University and a Guest Professor in the Wu Han University.
His research is mainly on applying statistics and mathematics for data analysis problems in data mining, computer vision, machine learning, multimedia, and visual surveillance. He has published around 100 scientific papers including IEEE TPAMI, TKDE, TIP, CVPR, ECCV, ICDM; ACM TKDD, Multimedia, KDD, etc., with best paper runner up awards and finalists. One of his TPAMI papers received an interview with ScienceWatch.com (Thomson Scientific). His H-Index in google scholar is 12 and his Erdös number is 3. Previously he gained several Meritorious Awards from the International Interdisciplinary Contest in Modeling, which is the highest level mathematical modeling contest in the world, organized by COMAP.
He is an associate editor of IEEE Transactions on Knowledge and Data Engineering, Neurocomputing (Elsevier) and the Official Journal of the International Association for Statistical Computing—Computational Statistics & Data Analysis (Elsevier). He is an editorial board member of Advances in Multimedia (Hindawi). He has authored/edited six books and eight journal special issues. He has (co-)chaired for special sessions, invited sessions, workshops, panels and conferences. He has served with more than 80 major international conferences including CVPR, ICCV, ECCV, ICDM, KDD, and Multimedia, and more than 30 prestigious international journals including TPAMI, TKDE, TOIS, and TIP. He is a member of IEEE, IEEE Computer Society, IEEE Signal Processing Society, IEEE SMC Society, and IEEE SMC Technical Committee on Cognitive Computing.
Yuan Yuan is currently a lecturer with the School of Engineering and Applied Science, Aston University, United Kingdom. She received her B.Eng. degree from the University of Science and Technology of China, China, and her Ph.D. degree from the University of Bath, United Kingdom. She has over 60 scientific publications in journals and conferences on visual information processing, compression, retrieval, etc. She is an associate editor of International Journal of Image and Graphics (World Scientific), an editorial board member of Journal of Multimedia (Academy Publisher), a guest editor of Signal Processing (Elsevier), and a guest editor of Recent Patents on Electrical Engineering. She was a chair of some conference sessions, and a member of program committees of many conferences. She is a reviewer for several IEEE transactions, other international journals and conferences. E-Mail: [email protected]
Xinbo Gao received his B.Sc., M.Sc. and Ph.D. degrees in Signal and Information Processing from the Xidian University, China, in 1994, 1997 and 1999, respectively. From 1997 to 1998, he was a research fellow in the Department of Computer Science at the Shizuoka University, Japan. From 2000 to 2001, he was a postdoctoral research fellow in the Department of Information Engineering at the Chinese University of Hong Kong. Since 2001, he joined the School of Electronic Engineering at the Xidian University. Currently, he is a Professor of Pattern Recognition and Intelligent System, and Director of the VIPS Lab, Xidian University. His research interests are computational intelligence, machine learning, computer vision, pattern recognition and artificial intelligence. In these areas, he has published 4 books and around 100 technical articles in refereed journals and proceedings including IEEE TIP, TCSVT, TNN, TSMC, etc. He is on the editorial boards of journals including EURASIP Signal Processing and Neurocomputing (Elsevier). He served as general chair/co-chair or program committee chair/co-chair or PC member for around 30 major international conferences. Email: [email protected]