Abstract
Image denoising methods are of fundamental importance in image processing and artificial intelligence systems. In this review, we analyze the traditional and state of the art mathematical models for computational color image denoising. These algorithms are divided into methods that are based on the partial differential equations, low rank, sparse representation and recent developments based on deep learning models. These algorithms also compared in terms of image quality measures. Our analysis and review of the computational color image denoising filters indicate that the convolutional neural networks from the deep learning domain obtain high quality restorations in terms of image quality despite the higher computational complexity.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Ambrosio L, Tortorelli VM (1990) Approximation of functional depending on jumps by elliptic functional via gamma-convergence. Commun Pure Appl Math 43(8):999–1036. https://doi.org/10.1002/cpa.3160430805
Astrom F, Schnorr C (2017) A geometric approach for color image regularization. Comput Vis Image Underst 165:43–59. https://doi.org/10.1016/j.cviu.2017.10.013
Bar L, Brook A, Sochen N, Kiryati N (2005) Color image deblurring with impulsive noise. In: Paragios N, Faugeras O, Chan T, Schnorr C (eds) Variational, geometric, and level set methods in computer vision. Springer, Berlin, pp 49–60
Barash D (2001) One-step deblurring and denoising color images using partial differential equations. Techcical report HPL2000, HP Labortries
Bevilacqua M, Roumy A, Guillemot C, Morel MA (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings of the British machine vision conference. BMVA Press, pp 135.1–135.10. https://doi.org/10.5244/C.26.135
Blomgren P, Chan TF (1998) Color TV: total variation methods for restoration of vector-valued images. IEEE Trans Image Process 7(3):304–309. https://doi.org/10.1109/83.661180
Boncelet C (2009) Chapter 7—image noise models. In: Bovik A (ed) The essential guide to image processing. Academic Press, Boston, pp 143–167. https://doi.org/10.1016/B978-0-12-374457-9.00007-X
Buades A, Coll B, Morel J (2005) A review of image denoising algorithms, with a new one. Multiscale Model Simul 4(2):490–530. https://doi.org/10.1137/040616024
Cha S, Moon T (2019) Fully convolutional pixel adaptive image denoiser. In: Proceedings of the IEEE international conference on computer vision, pp 4160–4169
Chan T, Shen J (2001) Variational restoration of nonflat image features: models and algorithms. SIAM J Appl Math 61(4):1338–1361. https://doi.org/10.1137/S003613999935799X
Chan TF, Kang SH, Shen J (2001a) Total variation denoising and enhancement of color images based on the CB and HSV color models. J Vis Commun Image Represent 12(4):422–435. https://doi.org/10.1006/jvci.2001.0491
Chan TF, Osher S, Shen J (2001b) The digital TV filter and nonlinear denoising. IEEE Trans Image Process 10:231–41. https://doi.org/10.1109/83.902288
Elad M, Aharon M (2006a) Image denoising via learned dictionaries and sparse representation. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR), vol 1, pp 895–900
Elad M, Aharon M (2006b) Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans Image Process 15(12):3736–3745
Franzen R (1999) Kodak lossless true color image suite. Available online at http://r0k.us/graphics/kodak/. Accessed 03 Mar 2021
Fuchs M, Muller J, Tietz C, Weickert J (2018) Convex regularization of multi-channel images based on variants of the TV-model. Complex Var Elliptic Equ 63(7–8):976–995. https://doi.org/10.1080/17476933.2017.1386181
Gai S (2019) Color image denoising via monogenic matrix-based sparse representation. Vis Comput 2019(35):109–122
Goyal B, Dogra A, Agrawal S, Sohi BS, Sharma A (2020) Image denoising review: from classical to state-of-the-art approaches. Inf Fusion 55:220–244. https://doi.org/10.1016/j.inffus.2019.09.003
Grecova S, Morillas S (2016) Perceptual similarity between color images using fuzzy metrics. J Vis Commun Image Represent 34:230–235. https://doi.org/10.1016/j.jvcir.2015.04.003
Gu S, Zhang L, Zuo W, Feng X (2014) Weighted nuclear norm minimization with application to image denoising. In: 2014 IEEE conference on computer vision and pattern recognition, pp 2862–2869
Guichard F, Moisan L, Morel JM (2002) A review of P.D.E. models in image processing and image analysis. J Phys IV 12:137–154. https://doi.org/10.1051/jp42002006
Guo S, Yan Z, Zhang K, Zuo W, Zhang L (2018) Toward convolutional blind denoising of real photographs. CoRR arxiv: abs/1807.04686
Hassan M, Bhagvati C (2012) Structural similarity measure for color images. Int J Comput Appl 43:7–12. https://doi.org/10.5120/6169-8590
Haykin S (1998) Neural networks: a comprehensive foundation, 2nd edn. Prentice Hall PTR, Upper Saddle River
He J, Zhang X, Ren S, Sun J (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T (eds) ECCV. Springer, Cham, pp 346–361
He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: IEEE international conference on computer vision (ICCV), vol 1502. https://doi.org/10.1109/ICCV.2015.123
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778
Hu Z, Huang Z, Huang X, Luo F, Ye R (2019) An adaptive nonlocal Gaussian prior for hyperspectral image denoising. IEEE Geosci Remote Sens Lett 16(9):1487–1491
Huang J, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: 2015 IEEE Conference on computer vision and pattern recognition (CVPR), pp 5197–5206. https://doi.org/10.1109/CVPR.2015.7299156
Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: 32nd International Conference on Machine Learning, PMLR 37. pp 448–456
Kim S (2006) PDE-based image restoration: a hybrid model and color image denoising. IEEE Trans Image Process 15(5):1163–1170. https://doi.org/10.1109/TIP.2005.864184
Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 1646–1654
Kingma D, Ba J (2014) Adam: a method for stochastic optimization. In: International conference on learning representations
Kolda TG, Bader BW (2009) Tensor decompositions and applications. SIAM Rev 51(3):455–500. https://doi.org/10.1137/07070111X
Kong Z, Yang X (2019) Color image and multispectral image denoising using block diagonal representation. IEEE Trans Image Process 28(9):4247–4259
Kong X, Zhao Y, Xue J, Chan JCW (2019) Hyperspectral image denoising using global weighted tensor norm minimum and nonlocal low-rank approximation. Remote Sens 11:2281
Krizhevsky A, Sutskever I, Hinton GE (2012a) Imagenet classification with deep convolutional neural networks. In: Proceedings of the 25th international conference on neural information processing systems, vol 1, NIPS. USA, pp 1097–1105
Krizhevsky A, Sutskever I, Hinton GE (2012b) Imagenet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in neural information processing systems, vol 25. Curran Associates, Inc., pp 1097–1105. http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
Lebrun M, Colom M, Morel JM (2015) The noise clinic: a blind image denoising algorithm. Image Process On Line 5:1–54. https://doi.org/10.5201/ipol.2015.125
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444. https://doi.org/10.1038/nature14539
Lee SH, Choi JY, Ro YM, Plataniotis KN (2012) Local color vector binary patterns from multichannel face images for face recognition. IEEE Trans Image Process 21(4):2347–2353
Lefkimmiatis S (2017) Non-local color image denoising with convolutional neural networks. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 5882–5891. https://doi.org/10.1109/CVPR.2017.623
Lehtinen J, Munkberg J, Hasselgren J, Laine S, Karras T, Aittala M, Aila T (2018) Noise2noise: Learning image restoration without clean data. CoRR arxiv: abs/1803.04189
Li X, Xiao J, Zhou Y, Ye Y, Lv N, Wang X, Wang S, Gao S (2020) Detail retaining convolutional neural network for image denoising. J Vis Commun Image Represent. https://doi.org/10.1016/j.jvcir.2020.102774
Liu P, Fang R (2017) Learning pixel-distribution prior with wider convolution for image denoising. CoRR arxiv: abs/1707.09135
Liu P, Zhang H, Zhang K, Lin L, Zuo W (2018) Multi-level wavelet-CNN for image restoration. CoRR arxiv: abs/1805.07071
Luo M, Cui G, Rigg B (2001) The development of the CIE 2000 colour-difference formula: Ciede 2000. Color Res Appl 26:340–350. https://doi.org/10.1002/col.1049
Lyu Z, Zhang C, Han M (2020) A nonsubsampled countourlet transform based CNN for real image denoising. Signal Process Image Commun 82(115):727. https://doi.org/10.1016/j.image.2019.115727
Ma Z, Wu HR, Feng D (2007) Fuzzy vector partition filtering technique for color image restoration. Comput Vis Image Understand 107(1):26–37. https://doi.org/10.1016/j.cviu.2006.11.017
Mairal J, Elad M, Sapiro G (2008) Sparse representation for color image restoration. IEEE Trans Image Process 17(1):53–69
Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings eighth IEEE international conference on computer vision. ICCV 2001, vol 2, pp 416–423. https://doi.org/10.1109/ICCV.2001.937655
Medioni G, Lee MS, Tang CK (2000) A computational framework for segmentation and grouping. Elsevier Science B.V., Amsterdam. https://doi.org/10.1016/B978-044450353-4/50002-0
Melange T, Nachtegael M, Kerre EE (2011) Fuzzy random impulse noise removal from color image sequences. IEEE Trans Image Process 20(4):959–970. https://doi.org/10.1109/TIP.2010.2077305
Moreno JC, Prasath VBS, Neves JC (2016a) Color image processing by vectorial total variation with gradient channels coupling. Inverse Probl Imaging 10(2):461–497. https://doi.org/10.3934/ipi.2016008
Moreno JC, Prasath VS, Santos G, Proença H (2016b) Robust periocular recognition by fusing sparse representations of color and geometry information. J Signal Process Syst 82(3):403–417. https://doi.org/10.1007/s11265-015-1023-3
Moreno R, Garcia MA, Puig D, Julia C (2011) Edge-preserving color image denoising through tensor voting. Comput Vis Image Understand 115(11):1536–1551. https://doi.org/10.1016/j.cviu.2011.07.005
Mukhopadhyay S, Mandal JK (2014) A fuzzy switching median filter of impulses in digital imagery (FSMF). Circuits Syst. Signal Process 33(7):2193–2216. https://doi.org/10.1007/s00034-014-9739-z
Mumford DB, Shah J (1989) Optimal approximations by piecewise smooth functions and associated variational problems. Communications on Pure and Applied Mathematics 42(5):577–685. https://doi.org/10.1002/cpa.3160420503
Pal C, Chakrabarti A, Ghosh R (2015) A brief survey of recent edge-preserving smoothing algorithms on digital images. CoRR arxiv: abs/1503.07297
Perona P, Malik J (1990) Scale-space and edge detection using anisotropic diffusion. IEEE Trans Pattern Anal Mach Intell 12(7):629–639. https://doi.org/10.1109/34.56205
Plötz T, Roth S (2017) Benchmarking denoising algorithms with real photographs. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 2750–2759. https://doi.org/10.1109/CVPR.2017.294
Prasath VBS, Singh A (2010) Multispectral image denoising by well-posed anisotropic diffusion scheme with channel coupling. Int J Remote Sens 31(8):2091–2099. https://doi.org/10.1080/01431160903260965
Prasath VS (2011) Weighted Laplacian differences based multispectral anisotropic diffusion. In: 2011 IEEE international geoscience and remote sensing symposium, pp 4042–4045. IEEE. https://doi.org/10.1109/IGARSS.2011.6050119
Prasath VS, Moreno JC (2018) On convergent finite difference schemes for variational–PDE-based image processing. Comput Appl Math 37(2):1562–1580. https://doi.org/10.1007/s40314-016-0414-9
Prasath VS, Vorotnikov D (2014) Weighted and well-balanced anisotropic diffusion scheme for image denoising and restoration. Nonlinear Anal Real World Appl 17:33–46. https://doi.org/10.1016/j.nonrwa.2013.10.004
Rafsanjani HK, Sedaaghi MH, Saryazdi S (2017) An adaptive diffusion coefficient selection for image denoising. Digit Signal Process 64:71–82. https://doi.org/10.1016/j.dsp.2017.02.004
Shah J (1996) A common framework for curve evolution, segmentation and anisotropic diffusion. In: Proceedings CVPR IEEE computer society conference on computer vision and pattern recognition, pp 136–142. https://doi.org/10.1109/CVPR.1996.517065
Shen Y, Han B, Braverman E (2016) Adaptive frame-based color image denoising. Appl Comput Harmonic Anal 41(1):54–74
Shi W, Jiang F, Zhang S, Wang R, Zhao D, Zhou H (2019) Hierarchical residual learning for image denoising. Signal Process Image Commun 76:243–251. https://doi.org/10.1016/j.image.2019.05.007
Sochen N, Kimmel R, Malladi R (1998) A general framework for low level vision. IEEE Trans Image Process 7(3):310–318. https://doi.org/10.1109/83.661181
Tian C, Xu Y, Li Z, Zuo W, Fei L, Liu H (2020a) Attention-guided CNN for image denoising. Neural Netw 124:117–129. https://doi.org/10.1016/j.neunet.2019.12.024
Tian C, Xu Y, Zuo W (2020b) Image denoising using deep CNN with batch renormalization. Neural Netw 121:461–473. https://doi.org/10.1016/j.neunet.2019.08.022
Tschumperle D (2006) Fast anisotropic smoothing of multi-valued images using curvature-preserving PDE’s. Int J Comput Vis 68:65–82. https://doi.org/10.1007/s11263-006-5631-z
Tschumperle D, Deriche R (2007) Anisotropic diffusion partial differential equations for multichannel image regularization: framework and applications. In: Hawkes PW (ed) Advances in imaging and electron Physics, vol 145. Elsevier, Amsterdam, pp 149–209
Turajlic E, Begovic A, Skaljo N (2019) Application of artificial neural network for image noise level estimation in the SVD domain. Electronics 8(2):20
Venetsanopoulos AN, Plataniotis KN (2000) Adaptive filters for color image processing: a survey. In: 2000 10th European signal processing conference, pp 1–4
Wald L (2002) Data fusion. Definitions and Architectures—Fusion of Images of Different Spatial Resolutions. Presses de l’Ecole, Ecole des Mines de Paris, Paris. https://hal-mines-paristech.archives-ouvertes.fr/hal-00464703. ISBN 2-911762-38-X
Wang Z, Simoncelli EP, Bovik AC (2003) Multiscale structural similarity for image quality assessment. In: The thrity-seventh Asilomar conference on signals, systems computers, vol 2, pp 1398–1402. https://doi.org/10.1109/ACSSC.2003.1292216
Wang T, Sun M, Hu K (2017a) Dilated residual network for image denoising. CoRR arxiv: abs/1708.05473
Wang Y, Yang Y, Chen T (2017b) Spectral-spatial adaptive and well-balanced flow-based anisotropic diffusion for multispectral image denoising. J Vis Commun Image Represent 43:185–197. https://doi.org/10.1016/j.jvcir.2017.01.005
Wei Y, Yuan Q, Shen H, Zhang L (2017) Boosting the accuracy of multispectral image pansharpening by learning a deep residual network. IEEE Geosci Remote Sens Lett 14(10):1795–1799
Wu X (2011) Color demosaicking by local directional interpolation and nonlocal adaptive thresholding. J Electron Imaging 20(2):023,016. https://doi.org/10.1117/1.3600632
Wyszecki G, Stiles W (2000) Color science: concepts and methods, quantitative data and formulae. In: Color science: concepts and methods, quantitative data and formulae, 2nd edn. Wiley, New York
Xie J, Xu L, Chen E (2012) Image denoising and inpainting with deep neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in neural information processing systems, vol 25. Curran Associates, Inc., pp 341–349. http://papers.nips.cc/paper/4686-image-denoising-and-inpainting-with-deep-neural-networks.pdf
Xie Q, Zhao Q, Meng D, Xu Z, Gu S, Zuo W, Zhang L (2016) Multispectral images denoising by intrinsic tensor sparsity regularization. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 1692–1700. https://doi.org/10.1109/CVPR.2016.187
Xie W, Li Y (2017) Hyperspectral imagery denoising by deep learning with trainable nonlinearity function. IEEE Geosci Remote Sens Lett 14(11):1963–1967
Xu Y, Yu L, Xu H, Zhang H, Nguyen T (2015) Vector sparse representation of color image using quaternion matrix analysis. IEEE Trans Image Process 24(4):1315–1329
Yu L, Xu Y, Xu H, Zhang H (2013) Quaternion-based sparse representation of color image. In: IEEE International conference on multimedia and expo (ICME), pp 1–7
Yuan Q, Zhang Q, Li J, Shen H, Zhang L (2019) Hyperspectral image denoising employing a spatial-spectral deep residual convolutional neural network. IEEE Trans Geosci Remote Sens 57(2):1205–1218
Zeyde R, Elad M, Protter M (2012) On single image scale-up using sparse-representations. In: Boissonnat JD, Chenin A, Gout C, Lyche T, Mazure ML, Schumaker L (eds) Curves and surfaces. Springer, Berlin, pp 711–730
Zhang L, Zhang L, Mou X, Zhang D (2011) FSIM: a feature similarity index for image quality assessment. IEEE Trans Image Process 20(8):2378–2386. https://doi.org/10.1109/TIP.2011.2109730
Zhang J, Zhao C, Xiong R, Ma S, Zhao D (2012) Image super-resolution via dual-dictionary learning and sparse representation. In: 2012 IEEE international symposium on circuits and systems (ISCAS), pp 1688–1691
Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017a) Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans Image Process 26(7):3142–3155. https://doi.org/10.1109/TIP.2017.2662206
Zhang K, Zuo W, Zhang L (2017b) Ffdnet: Toward a fast and flexible solution for CNN based image denoising. CoRR. arxiv: abs/1710.04026
Zhang Q, Yuan Q, Li J, Liu X, Shen H, Zhang L (2019) Hybrid noise removal in hyperspectral imagery with a spatial–spectral gradient network. IEEE Trans Geosci Remote Sens 57(10):7317–7329
Funding
Funding was provided by National Center for Advancing Translational Sciences (Grant No. U2CTR002818), National Heart, Lung, and Blood Institute (Grant No. U24HL148865), National Institute of Allergy and Infectious Diseases (Grant No. U01AI150748), Cincinnati Children's Hospital Medical Center (Grant No. Advanced Research Council (ARC) Grant 2018-2020).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Salamat, N., Missen, M.M.S. & Surya Prasath, V.B. Recent developments in computational color image denoising with PDEs to deep learning: a review. Artif Intell Rev 54, 6245–6276 (2021). https://doi.org/10.1007/s10462-021-09977-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10462-021-09977-z