Abstract
Noise is the most common type of image distortion affecting human visual perception. In this paper, we propose a no-reference image quality assessment (IQA) method for noisy images incorporating the features of entropy, gradient, and kurtosis. Specifically, image noise estimation is conducted in the discrete cosine transform domain based on skewness invariance. In the principal component analysis domain, kurtosis feature is obtained by statistically counting the significant differences between images with and without noise. In addition, both the consistency between the entropy and kurtosis features and the subjective scores are improved by combining them with the gradient coefficient. Support vector regression is applied to map all extracted features into an integrated scoring system. The proposed method is evaluated in three mainstream databases (i.e., LIVE, TID2013, and CSIQ), and the results demonstrate the superiority of the proposed method according to the Pearson linear correlation coefficient which is the most significant indicator in IQA.
摘要
噪声是影响人类视觉感知最常见的图像失真类型. 本文提出一种基于熵、 梯度和峰度特征的无参考图像质量评估方法. 具体来说, 基于偏度不变性在离散余弦变换域进行图像噪声估计, 进一步计算得到熵特征. 在主成分分析变换域, 通过统计有噪声图像与无噪声图像之间的显著差异得到峰度特征. 此外, 将熵和峰度特征与梯度系数结合, 提高熵和峰度特征与主观得分之间的一致性. 通过不同方向的滤波器对图像进行梯度特征提取, 最后支持向量回归将所有提取的特征映射到综合评分系统中. 为验证算法性能, 在3个主流数据库 (即LIVE、 TID2013以及CSIQ) 中对该方法进行评价. 实验结果验证了该方法的优越性, 尤其是在反映预测精度的皮尔逊线性相关系数方面的突出性能.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bosse S, Maniry D, Wiegand T, et al., 2016. A deep neural network for image quality assessment. Proc IEEE Int Conf on Image Processing, p.3773–3777. https://doi.org/10.1109/ICIR2016.7533065
Buczkowski M, 2018. Non-reference image quality assessment based on noise estimation. Proc 25th Int Conf on Systems, Signals and Image Processing, p.1–4. https://doi.org/10.1109/IWSSIP.2018.8439331
Chang CC, Lin CJ, 2011. LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol, 2(3):27. https://doi.org/10.1145/1961189.1961199
Chen DQ, Wang YZ, Gao W, 2020. No-reference image quality assessment: an attention driven approach. IEEE Trans Image Process, 29:6496–6506. https://doi.org/10.1109/TIP.2020.2990342
Deng CW, Wang SG, Bovik AC, et al., 2020. Blind noisy image quality assessment using sub-band kurtosis. IEEE Trans Cybern, 50(3):1146–1156. https://doi.org/10.1109/TCYB.2018.2889376
Ding Y, Li N, Zhao Y, et al., 2016. Image quality assessment method based on nonlinear feature extraction in kernel space. Front Inform Technol Electron Eng, 17(10):1008–1017. https://doi.org/10.1631/FITEE.1500439
Dong L, Zhou JT, Tang YY, 2017. Noise level estimation for natural images based on scale-invariant kurtosis and piecewise stationarity. IEEE Trans Image Process, 26(2): 1017–1030. https://doi.org/10.1109/TIP.2016.2639447
Gu K, Zhai GT, Yang XK, et al., 2015. Using free energy principle for blind image quality assessment. IEEE Trans Multim, 17(1):50–63. https://doi.org/10.1109/TMM.2014.2373812
Guo R, Shen XJ, Dong XY, et al., 2020. Multi-focus image fusion based on fully convolutional networks. Front Inform Technol Electron Eng, 21(7):1019–1033. https://doi.org/10.1631/FITEE.1900336
Hu B, Li LD, Wu JJ, et al., 2020. Subjective and objective quality assessment for image restoration: a critical survey. Signal Process Image Commun, 85:115839. https://doi.org/10.1016/jimage.2020.115839
Huang XT, Chen L, Tian J, et al., 2014. Blind noisy image quality assessment using block homogeneity. Comput Electr Eng, 40(3):796–807. https://doi.org/10.1016/jxompeleceng.2013.08.002
Jiang XH, Shen LQ, Yu LW, et al., 2020. No-reference screen content image quality assessment based on multi-region features. Neurocomputing, 386:30–41. https://doi.org/10.1016/j.neucom.2019.12.027
Kennedy J, Eberhart R, 1995. Particle swarm optimization. Proc Int Conf on Neural Networks, p.1942–1948. https://doi.org/10.1109/ICNN.1995.488968
Kong XF, Li K, Yang QX, et al., 2013. A new image quality metric for image auto-denoising. Proc IEEE Int Conf on Computer Vision, p.2888–2895. https://doi.org/10.1109/ICCV.2013.359
Larson EC, Chandler DM, 2010. Most apparent distortion: full-reference image quality assessment and the role of strategy. J Electron Image, 19(1):011006. https://doi.org/10.1117/1.3267105
Li LD, Xia WH, Fang YM, et al., 2016a. Color image quality assessment based on sparse representation and reconstruction residual. J Vis Commun Image Represent, 38: 550–560. https://doi.org/10.1016/j.jvcir.2016.04.006
Li LD, Lin WS, Wang XS, et al., 2016b. No-reference image blur assessment based on discrete orthogonal moments. IEEE Trans Cybern, 46(1):39–50. https://doi.org/10.1109/TCYB.2015.2392129
Li LD, Xia WH, Lin WS, et al., 2017. No-reference and robust image sharpness evaluation based on multiscale spatial and spectral features. IEEE Trans Multim, 19(5): 1030–1040. https://doi.org/10.1109/TMM.2016.2640762
Li PY, Lo KT, 2018. A content-adaptive joint image compression and encryption scheme. IEEE Trans Multim, 20(8): 1960–1972. https://doi.org/10.1109/TMM.2017.2786860
Li QH, Lin WS, Fang YM, 2017. BSD: blind image quality assessment based on structural degradation. Neurocomputing, 236:93–103. https://doi.org/10.1016/j.neucom.2016.09.105
Liu M, Zhai GT, Zhang ZY, et al., 2014. Blind image quality assessment for noise. Proc IEEE Int Symp on Broadband Multimedia Systems and Broadcasting, p.1–5. https://doi.org/10.1109/BMSB.2014.6873480
Lyu SW, Pan XY, Zhang X, 2014 Exposing region splicing forgeries with blind local noise estimation. Int J Comput Vis, 110(2):202–221. https://doi.org/10.1007/s11263-013-0688-y
Ma B, Yao JC, Le YF, et al., 2020. Efficient image noise estimation based on skewness invariance and adaptive noise injection. IET Image Process, 14(7):1393–1401. https://doi.org/10.1049/iet-ipr.2019.1548
Min XK, Zhai GT, Gu K, et al., 2018. Blind image quality estimation via distortion aggravation. IEEE Trans Broadcast, 64(2):508–517. https://doi.org/10.1109/TBC.2018.2816783
Mittal A, Moorthy AK, Bovik AC, 2012. No-reference image quality assessment in the spatial domain. IEEE Trans Image Process, 21(12):4695–4708. https://doi.org/10.1109/TIP.2012.2214050
Mittal A, Soundararajan R, Bovik AC, 2013. Making a “completely blind” image quality analyzer. IEEE Signal Process Lett, 20(3):209–212. https://doi.org/10.1109/LSP.2012.2227726
Moorthy AK, Bovik AC, 2011. Blind image quality assessment: from natural scene statistics to perceptual quality. IEEE Trans Image Process, 20(12):3350–3364. https://doi.org/10.1109/TIP.2011.2147325
Ospina-Borras JE, Restrepo HDB, 2016. Non-reference assessment of sharpness in blur/noise degraded images. J Vis Commun Image Represent, 39:142–151. https://doi.org/10.1016/j.jvcir.2016.05.015
Oszust M, 2019. No-reference quality assessment of noisy images with local features and visual saliency models. Inform Sci, 482:334–349. https://doi.org/10.1016/j.ins.2019.01.034
Pan CH, Xu Y, Yan YC, et al., 2016. Exploiting neural models for no-reference image quality assessment. Proc Visual Communications and Image Processing, p.1–4. https://doi.org/10.1109/VCIP.2016.7805524
Ponomarenko N, Ieremeiev O, Lukin V, et al., 2013. A new color image database TID2013: innovations and results. Proc 15th Int Conf on Advanced Concepts for Intelligent Vision Systems, p.402–413. https://doi.org/10.1007/978-3-319-02895-8_36
Saad MA, Bovik AC, Charrier C, 2012. Blind image quality assessment: a natural scene statistics approach in the DCT domain. IEEE Trans Image Process, 21(8):3339–3352. https://doi.org/10.1109/TIP.2012.2191563
Sheikh HR, Sabir MF, Bovik AC, 2006. A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans Image Process, 15(11):3440–3451. https://doi.org/10.1109/TIP.2006.881959
Shen LL, Hang N, Hou CP, 2020. Feature-segmentation strategy based convolutional neural network for no-reference image quality assessment. Multim Tool Appl, 79(17–18):11891–11904. https://doi.org/10.1007/s11042-019-08298-2
Tang LJ, Li LD, Sun KZ, et al., 2017. An efficient and effective blind camera image quality metric via modeling quaternion wavelet coefficients. J Vis Commun Image Represent, 49:204–212. https://doi.org/10.1016/j.jvcir.2017.09.010
Tang ZJ, Huang ZQ, Yao H, et al., 2018. Perceptual image hashing with weighted DWT features for reduced-reference image quality assessment. Comput J, 61(11): 1695–1709. https://doi.org/10.1093/comjnl/bxy047
tv2). http://www.vqeg.org
Wang Q, Chu J, Xu L, et al., 2016. A new blind image quality framework based on natural color statistic. Neurocomputing, 173:1798–1810. https://doi.org/10.1016/j.neucom.2015.09.057
Wang Z, Bovik AC, Sheikh HR, et al., 2004. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process, 13(4):600–612. https://doi.org/10.1109/TIP.2003.819861
Wu JJ, Zhang M, Li LD, et al., 2019. No-reference image quality assessment with visual pattern degradation. Inform Sci, 504:487–500. https://doi.org/10.1016/j.ins.2019.07.061
Xu L, Huang G, Chen QL, et al., 2020. An improved method for image denoising based on fractional-order integration. Front Inform Technol Electron Eng, 21(10):1485–1493. https://doi.org/10.1631/FITEE.1900727
Ye P, Kumar J, Kang L, et al., 2012. Unsupervised feature learning framework for no-reference image quality assessment. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.1098–1105. https://doi.org/10.1109/CVPR.2012.6247789
Zhai GT, Wu XL, 2011. Noise estimation using statistics of natural images. Proc 18th IEEE Int Conf on Image Processing, p.1857–1860. https://doi.org/10.1109/ICIP.2011.6115828
Zhai GT, Wu XL, Yang XK, et al., 2012. A psychovisual quality metric in free-energy principle. IEEE Trans Image Process, 21(1):41–52. https://doi.org/10.1109/TIP.2011.2161092
Zhai GT, Kaup A, Wang J, et al., 2015. A dual-model approach to blind quality assessment of noisy images. APSIPA Trans Signal Inform Process, 4:e4. https://doi.org/10.1017/ATSIP.2015.8
Zhang L, Zhang L, Bovik AC, 2015. A feature-enriched completely blind image quality evaluator. IEEE Trans Image Process, 24(8):2579–2591. https://doi.org/10.1109/TIP.2015.2426416
Zhou WJ, Yu L, Qiu WW, et al., 2017. Local gradient patterns (LGP): an effective local-statistical-feature extraction scheme for no-reference image quality assessment. Inform Sci, 397–398:1–14. https://doi.org/10.1016/j.ins.2017.02.049
Zhu HC, Li LD, Wu JJ, et al., 2020. MetaIQA: deep metalearning for no-reference image quality assessment. IEEE/CVF Conf on Computer Vision and Pattern Recognition, p. 14143–14152. https://doi.org/10.1109/CVPR42600.2020.01415
Zhu T, Karam L, 2014. A no-reference objective image quality metric based on perceptually weighted local noise. EURASIP J Image Video Process, 2014(1):1–8. https://doi.org/10.1186/1687-5281-2014-5
Zoran D, Weiss Y, 2009. Scale invariance and noise in natural images. Proc IEEE Int Conf on Computer Vision, p.2209–2216. https://doi.org/10.1109/ICCV.2009.5459476
Author information
Authors and Affiliations
Contributions
Heng YAO designed the algorithms. Ben MA drafted the manuscript. Mian ZOU processed the data. Dong XU and Jincao YAO supervised the algorithms, and revised and finalized the paper.
Corresponding authors
Ethics declarations
Heng YAO, Ben MA, Mian ZOU, Dong XU, and Jincao YAO declare that they have no conflict of interest.
Additional information
Project supported by the National Natural Science Foundation of China (No. 61702332) and the Zhejiang Provincial Natural Science Foundation of China (Nos. LZY21F030001 and LSD19H180001)
Heng YAO, first author of this invited paper, received his BS degree from Hefei University of Technology, China, in 2004, his MS degree from Shanghai Normal University, China, in 2008, and his PhD degree in Signal and Information Processing from Shanghai University, China, in 2012. He is currently an associate professor at University of Shanghai for Science and Technology. His research interests include multimedia security, image processing, and pattern recognition. He has published more than 40 peer-reviewed papers in international journals.
Dong XU, corresponding author of this invited paper, received his BS and MS degrees in Medical Imaging from Southeast University and Zhejiang University in 2002 and 2008, respectively. In 2011, he received his PhD degree in Clinical Specialty of Integrated Traditional Chinese and Western Medicine from Zhejiang Chinese Medical University. He is now a professor at the Institute of Basic Medicine and Cancer (IBMC), Chinese Academy of Sciences and Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital). His current research interests include medical image processing, machine learning, radiomics, and medical imaging diagnosis of tumor.
Jincao YAO, corresponding author of this invited paper, received his BS degree in Computer and Information Science from Hefei University of Technology in 2004, and his MS degree in Signal Processing from Shanghai Normal University in 2007. In 2017, he received his PhD degree in Signal and Information Processing at the College of Information Science and Electronic Engineering, Zhejiang University, China. He is now an imaging physicist at the IBMC, Chinese Academy of Sciences and Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital). His current research interests include machine learning, computer vision, medical image recognition, radiomics, and shape-driven techniques in image processing.
Rights and permissions
About this article
Cite this article
Yao, H., Ma, B., Zou, M. et al. No-reference noisy image quality assessment incorporating features of entropy, gradient, and kurtosis. Front Inform Technol Electron Eng 22, 1565–1582 (2021). https://doi.org/10.1631/FITEE.2000716
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/FITEE.2000716