Skip to main content
SpringerLink
Log in

RETRACTED ARTICLE: Lightweight deep dense Demosaicking and Denoising using convolutional neural networks

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

This article was retracted on 13 September 2022

This article has been updated

Abstract

A single sensor camera uses Color Filter Array (CFA) to capture single-color information at each pixel. Thus, to estimate the missing color samples and then to reconstruct an original image is known as CFA interpolation or demosaicking. Despite remarkable improvements made in the last decade, a fundamental issue remains to be addressed, i.e., how to assure the visual quality of an image in the presence of noise. Hence, the CFA images without denoising lead to the demosaicking artifacts that eventually reduce the image quality. Therefore, based on the aforementioned constraints, the paper presents a novel approach for demosaicking and denoising based on the convolutional neural network (CNN). The proposed technique is using CNN, which consists of four phases. In the first stage, the picture is sorted out. In stage-II, the demosaicking is performed utilizing the profound thick convolutional neural system, which gives us a demosaicked picture. In the stage-III, denoising performs and pass this picture to the last stage. At last, in the stage-IV, the picture goes to the last post-preparing stage delivering a better quality high-resolution image. To test the feasibility of the proposed scheme, Python language is utilized. The proposed conspire beats the few existing strategies regarding throughput delay, inactivity, precision.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Change history

References

  1. Adams JE Jr (Feb. 1997) Design of practical color filter array interpolation algorithms for digital cameras. Proc SPIE 3028:117–125

    Article  Google Scholar 

  2. Alleysson D, Susstrunk S, Herault J (April 2005) Linear demosaicing inspired by the human visual system. IEEE Trans Image Process 14(4):439–449

    Article  Google Scholar 

  3. Bonanomi C, Balletti S, Lecca M, Anisetti M, Rizzi A, Damiani E (2018) I3D: a new dataset for testing denoising and demosaicing algorithms. Multimed Tools Appl:1–28

  4. Buades A, Coll B, Morel J-M, Sbert C (2009) Self-similarity driven color demosaicking. IEEE Trans Image Process 18(6):1192–1202

    Article  MathSciNet  Google Scholar 

  5. Chang E, Cheung S, Pan DY (Jan. 1999) Color filter array recovery using a threshold-based variable number of gradients. Proc SPIE 3650:36–43

    Article  Google Scholar 

  6. Chang K, Ding PLK, Li B (2015) Color image demosaicking using inter-channel correlation and nonlocal self-similarity. Signal Process Image Commun 39:264–279

    Article  Google Scholar 

  7. Cok DR (1994) Reconstruction of CCD images using template matching. Proc IS&T Annual Conf / ICPS:380–385

  8. Dong W, Yuan M , Li X, Shi G (2018) Joint demosaicing and denoising with perceptual optimization on a generative adversarial network. [online] Available: https://arxiv.org/abs/1802.04723.

  9. Duran J, Buades A (2014) Self-similarity and spectral correlation adaptive algorithm for color demosaicking. IEEE Trans Image Process 23(9):4031–4040

    Article  MathSciNet  Google Scholar 

  10. Ehrhardt MJ, Arridge SR (Jan. 2014) Vector-valued image processing by parallel level sets. IEEE Trans Image Process 23(1):9–18

    Article  MathSciNet  Google Scholar 

  11. Foi A, Trimeche M, Katkovnik V, Egiazarian K (Oct 2008) Practical Poissonian-gaussian noise modeling and fitting for single-image rawdata. IEEE Trans Image Process 17(10):1737–1754

    Article  MathSciNet  Google Scholar 

  12. Gharbi M, Chaurasia G, Paris S, Durand F (Nov. 2016) Deep joint demosaicking and denoising. ACM Trans Graph 35(6):191:1–191:12

    Article  Google Scholar 

  13. Go J, Lee C (June 2000) Interpolation using neural network for digital still cameras, pp. 176–177

  14. Go J, Sohn K, Lee C (2000) Interpolation using neural networks for digital still cameras. IEEE Trans Consum Electron 46(3):610–616

    Article  Google Scholar 

  15. Gunturk BK, Altunbasak Y, Mersereau RM (Sept. 2002) Color plane interpolation using alternating projections. IEEE Trans Image Process 11:997–1013

    Article  Google Scholar 

  16. He F-L, Wang Y-CF, Hua K-L (2012) Self-learning approach to color demosaicking via support vector regression. Image Processing (ICIP), 2012 19th IEEE International Conference on IEEE:2765–2768

  17. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition:770–778

  18. He K, Zhang X, Ren S, Sun J (June 2016) Deep residual learning for image recognition. Proc IEEE Conf Comput Vis Pattern Recognit, Vegas, pp 770–778

    Google Scholar 

  19. Heide F, Steinberger M, Tsai Y-T, Rouf M, Pajak D, Reddy D, Gallo O, Liu J, Heidrich W, Egiazarian K et al (2014) Flexisp: a flexible camera image processing framework. ACM Transactions on Graphics (TOG) 33(6):231

    Article  Google Scholar 

  20. Helor HZ (2000) Demosaicking using artificial neural networks. Proc SPIE 45(3962):112–120

    Google Scholar 

  21. Henz BE, Gastal SL, Oliveira MM (2018) Deep joint design of color filter arrays and demosaicing. Computer Graphics Forum, vol. 37, no. 2, pp. 389–399. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1111/cgf.13370.

  22. Hirakawa K, Parks TW (Aug. 2006) Joint demosaicking and denoising. IEEE Trans Image Process 15(8):2146–2157

    Article  Google Scholar 

  23. Hirakawa K, Meng X-L, Wolfe PJ (Apr. 2007) A framework for waveletbased analysis and processing of color filter array images with applications to denoising and demosaicing. Proc ICASSP 1:I-597–I-600

  24. Iizuka S, Simo-Serra E, Ishikawa H (2016) Let there be color!: joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification. ACM Trans Graph 35(4):110

    Article  Google Scholar 

  25. Khashabi D, Nowozin S, Jancsary J, Fitzgibbon AW (Dec 2014) Joint Demosaicing and Denoising via learned nonparametric random fields. IEEE Trans Image Process 23(12):4968–4981

    Article  MathSciNet  Google Scholar 

  26. Koh CC, Mukherjee J, Mitra SK (Nov. 2003) New efficient methods of image compression in digital cameras with color filter array. IEEE Trans Consum Electron 49(11):1448–1456

    Google Scholar 

  27. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems 25: 26th Annual Conference on Neural Information Processing Systems. Proceedings of a meeting held December, vol 3–6.VLake Tahoe, Nevada, United States, pp 1106–1114

  28. Long J, Shelhamer E, Darrell T (June 7-12, 2015) Fully convolutional networks for semantic segmentation. IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp 3431–3440

  29. Lukac R, Plataniotis KN (2006) A taxonomy of color image filtering and enhancement solutions. In: Hawkes PW (ed) Advances in imaging and electron physics, vol 140. Elsevier/Academic, New York, pp 187–264

    Google Scholar 

  30. Lukac R, Smolka B, Martin K, Plataniotis KN, Venetsanopoulos AN (Jan. 2005) Vector filtering for color imaging. IEEE Signal Process Mag 22(1):74–86

  31. Mizutani J, Ogawa S, Shinoda K, Hasegawa M, Kato S (2014) Multispectral, demosaicking algorithm based on inter-channel correlation. Visual Communications and Image Processing Conference, IEEE

    Book  Google Scholar 

  32. Noh H, Hong S, Han B (December 7–13, 2015) Learning deconvolution network for semantic segmentation. 2015 IEEE International Conference on Computer Vision, ICCV, pp 1520–1528

  33. Oren K, Hel-Or HZ (2000) Demosaicking using artificial neural networks. Proc.SPIE, vol. 3962, pp. 9. [Online]. Available: https://doi.org/10.1117/12.382904

  34. Paliy D, Katkovnik V, Bilcu R, Alenius S, Egiazarian K (2007) Spatially adaptive color filter array interpolation for noiseless and noisy data. Int J Imaging Systems and Technology, Special Issue on Applied Color Image Processing 17:105–122

    Google Scholar 

  35. Paliy D, Trimeche M, Katkovnik V, Alenius S (2007) Demosaicing of noisy data: spatially adaptive approach. Proc SPIE 6497:64970K–64970K

    Article  Google Scholar 

  36. Ramanath R, Snyder WE (Oct. 2003) Adaptive demosaicking. J Electron Imag 12(4):633–642

    Article  Google Scholar 

  37. Ramanath R, Snyder WE, Bilbro GL (July 2002) Demosaicking methods for Bayer color arrays. J Electronic Imaging 11:306–315

    Article  Google Scholar 

  38. Shen W, Wang X, Wang Y, Bai X, Zhang Z (June 7-12, 2015) Deepcontour: a deep convolutional feature learned by positive-sharing loss for contour detection, vol 2015. IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, pp 3982–3991

  39. Shi W, Caballero J, Huszar F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video superresolution using an efficient sub-pixel convolutional neural network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition:1874–1883

  40. Sun J, Tappen MF (Jan 2013) Separable Markov random field model and its applications in low level vision. IEEE Trans Image Process 22(1):402–407

    Article  MathSciNet  Google Scholar 

  41. Tan H, Zeng X, Lai S, Liu Y, Zhang M (2017) Joint demosaicing and denoising of noisy bayer images with ADMM. IEEE International Conference on Image Processing

  42. Tan DS, Chen W, Hua K (May 2018) Deepdemosaicking: adaptive image demosaicking via multiple deep fully convolutional networks. IEEE Trans Image Process 27(5):2408–2419

    Article  MathSciNet  Google Scholar 

  43. Tan DS, Chen W-Y, Hua K-L (07 February 2018) Deepdemosaicking: adaptive image demosaicking via multiple deep fully convolutional networks. IEEE Transactions on Image Processing:2408–2419

  44. Trussell HJ, Hartwig RE (Apr. 2002) Mathematics for demosaicking. IEEE Trans Image Process 11(4):485–492.

  45. Wang Y (Oct 2014) A multilayer neural network for image demosaicking pp. 1852–1856

  46. Wu X, Zhang N (Sept. 2003) Primary-consistent soft-decision color demosaic for digital cameras. Proc:ICIP: I:477–ICIP I:480

  47. Xie S, Tu Z (December 7–13, 2015) Holistically-nested edge detection. 2015 IEEE International Conference on Computer Vision, ICCV, pp 1395–1403

  48. Xin Li LZ, Gunturk B (2008) Image demosaicing: a systematic survey. 6822:15

  49. Zhang L, Wu X (Dec. 2005) Color demosaicking via directional linear minimum mean square-error estimation. IEEE Trans Image Process 14(12):2167–2178

    Article  Google Scholar 

  50. Zhang L, Wu X, Zhang D (Sep. 2007) Color reproduction from noisy CFA data of single sensor digital cameras. IEEE Trans Image Process 16(9):2184–2197

    Article  MathSciNet  Google Scholar 

  51. Zhang L, Wu X, Buades A, Li X (2011) Color demosaicking by local directional interpolation and nonlocal adaptive thresholding. J Electron Imaging 20(2):023016

    Article  Google Scholar 

  52. Zhang X, Sun M-T, Fang L, Oscar CAU (2014) Joint denoising and demosaicking of noisy CFA images based on inter-color correlation vol 2014. IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP), pp 474–477

    Google Scholar 

  53. Zhang R, Isola P, Efros AA (2016) Colorful image colorization. In Computer Vision - ECCV - 14th European Conference, vol October 11-14, Amsterdam, The Netherlands, pp 649–666

  54. Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017) Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing

Download references

Acknowledgments

This work is supported by BK21 Plus project (SW Human Resource Development Program for Supporting Smart Life) funded by the Ministry of Education, School of Computer Science and Engineering, Kyungpook National University, Korea (21A20131600005).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Kyungpook National University, Daegu, South Korea

    Sadia Din & Anand Paul

  2. Dipartimento di Informatica, Università Degli Studi di Milano, Milan, Italy

    Awais Ahmad

Authors
  1. Sadia Din
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anand Paul
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Awais Ahmad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anand Paul.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article has been retracted. Please see the retraction notice for more detail: https://doi.org/10.1007/s11042-022-13875-z

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Din, S., Paul, A. & Ahmad, A. RETRACTED ARTICLE: Lightweight deep dense Demosaicking and Denoising using convolutional neural networks. Multimed Tools Appl 79, 34385–34405 (2020). https://doi.org/10.1007/s11042-020-08908-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08908-4

Keywords

Search

Navigation