Skip to main content
SpringerLink
Log in

Bayer image demosaicking and denoising based on specialized networks using deep learning

  • Special Issue Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Demosaicking is the way toward reproducing a full hued picture from a deficient shaded picture. The single sensor doesn't catch all hues for a single pixel. To address this, a color filter array (CFA) is utilized to get a hued picture from a single sensor. The created picture from CFA is called a mosaic picture. In this research, we utilize specialized networks to remove the noise from Bayer images. The mosaic picture is adulterated by commotion presented by a sensor or other equipment during catching. Demosaicking on the boisterous mosaic picture makes antiquities, for example, moiré and zippering. Some solutions have been proposed for denoising mosaic images but they are handcrafted solutions. In this paper, a solution is proposed to the first denoise and then demosaic the image using machine learning. The mosaic image is denoised using CNN which is then demosaicked using the residual learning strategy of a single specialized network. One of the networks is DHTN (deep high textured network) which is trained on textured images and the second one is DSTN (deep smooth textured network) which is trained on smooth images. Preliminary results show that the proposed approach generates better results and higher quality images than traditional approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
FIG. 8.
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. R. Tan, K. Zhang, W. Zuo and L. Zhang, "Color image demosaicking via deep residual learning," in IEEE Int. Conf. Multimedia Expo (ICME), 2017

  2. Zhang, L., Wu, X.: Color demosaicking via directional linear minimum mean square-error estimation. IEEE Trans. Image Process. 14(12), 2167–2178 (2005)

    Article  Google Scholar 

  3. M. Gharbi, G. Chaurasia, S. Paris and F. Durand, "Deep joint demosaicking and denoising," ACM Transactions on Graphics (TOG), 2016.

  4. D. S. Tan, W.-Y. Chen and K.-L. Hua, Deep Demosaicking Adaptive Image Demosaicking via Multiple Deep Fully Convolutional Networks. In: IEEE Transactions on Image Processing, pp. 2408–2419, 2018.

  5. D. Kiku, Y. Monno, M. Tanaka and M. Okutomi, Residual interpolation for color image demosaicking, 2013 IEEE International Conference on Image Processing, 2014.

  6. Menon, D., Andriani, S., Calvagno, G.: Demosaicing with directional filtering and a posteriori decision. IEEE Trans. Image Process. 16(1), 132–141 (2007)

    Article  MathSciNet  Google Scholar 

  7. Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2010)

    Article  Google Scholar 

  8. L. Condat and S. Mosaddegh, Joint demosaicking and denoising by total variation minimization, in 2012 19th IEEE International Conference on Image Processing, 2012.

  9. C. U. S. B. C. U. P. Longere Dept. of Psychol., X. Zhang, P. Delahunt and D. Brainard, Perceptual assessment of demosaicing algorithm performance, in Proceedings of the IEEE , 2002.

  10. J. E. Adams, Intersections between color plane interpolation and other image processing functions in electronic photography, Cameras and Systems for Electronic Photography and Scientific Imaging, vol. 2416, 1995.

  11. J. F. H. J. James E. Adams, Adaptive Color Plane Interpolation in Single Color Electronic Camera, US5652621A,1996.

  12. E. Chang, S. Cheung and D. Pan, Color filter array recovery using a threshold-based variable number of gradients, in Proceedings of SPIE—the international society for optical engineering, 1999.

  13. Kimmel, R.: Demosaicing: image reconstruction from CCD samples. IEEE Trans. Image Process. 8(9), 1221–1228 (1999)

    Article  Google Scholar 

  14. Gunturk, B., Altunbasak, Y., Mersereau, R.: Color plane interpolation using alternating projections. IEEE Trans. Image Process. 11(9), 997–1013 (2002)

    Article  Google Scholar 

  15. Gunturk, B.K., Glotzbach, J., Altunbasak, Y., Schafer, R.W., Mersereau, R.M.: Demosaicking: color filter array interpolation. IEEE Signal Process. Mag. 22(1), 44–54 (2005)

    Article  Google Scholar 

  16. Hirakawa, K., Parks, T.: Adaptive homogeneity-directed demosaicing algorithm. IEEE Trans. Image Process. 13(3), 360–369 (2005)

    Article  Google Scholar 

  17. Lukac, R., Plataniotis, K.: Color filter arrays: design and performance analysis. IEEE Trans. Consum. Electron. 51(4), 1260–1267 (2005)

    Article  Google Scholar 

  18. Zhang, L., Lukac, R., Wu, X., Zhang, D.: PCA-based spatially adaptive denoising of CFA images for single-sensor digital cameras. IEEE Trans. Image Process. 18(4), 797–812 (2009)

    Article  MathSciNet  Google Scholar 

  19. Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)

    Article  MathSciNet  Google Scholar 

  20. Kiku, D., Monno, Y., Tanaka, M., Okutomi, M.: Beyond color difference residual interpolation. IEEE Trans. Image Process. 25(3), 1288–1300 (2016)

    MathSciNet  MATH  Google Scholar 

  21. D. Alleysson, S. Süsstrunk and J. Hérault, Linear demosaicing inspired by the human visual system, IEEE Transactions on Image Processing, vol. 14, no. 4, 2005.

  22. X. Chen, H. Wu, B. Yang and G. Geon, Demosaicking based on dual-CFA pattern, in 2017 IEEE International Conference on Imaging Systems and Techniques (IST), 2017.

  23. Bai, C., Li, J., Lin, Z., Yu, J.: Automatic design of color filter arrays in the frequency domain. IEEE Trans. Image Process. 25(4), 1793–1807 (2016)

    MathSciNet  MATH  Google Scholar 

  24. Y. Monno, D. Kiku, M. Tanaka and M. Okutomi, Adaptive residual interpolation for color image demosaicking, in 2015 IEEE International Conference on Image Processing (ICIP), 2015.

  25. N. Li, J. S. J. Li, S. Randhawa and D. G. Bailey, Edge Preserving CFA Demosaicking based on Nonlinear Weighted Color Differences, in 2016 IEEE Region 10 Conference (TENCON), 2016.

  26. K. He, X. Zhang, S. Ren and J. Sun, Deep residual learning for image recognition, in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, 2016, pp. 770–778.

  27. Khashabi, D., Nowozin, S., Jancsary, J., Fitzgibbon, A.W.: Joint demosaicing and denoising via learned non-parametric random fields. IEEE Trans. Image Process. 23(12), 4968–4981 (2014)

    Article  MathSciNet  Google Scholar 

  28. Zhang, L., Wub, X., Buadesc, A., Lid, X.: Color demosaicking by local directional interpolation and nonlocal adaptive thresholding. J. Electron. Imaging 20(2), 023016 (2011). https://doi.org/10.1117/1.3600632

    Article  Google Scholar 

  29. Zhang, F., Wu, X., Yang, X., Zhang, W., Zhang, L.: Robust color demosaicking with adaptation to varying spectral correlations. IEEE Trans. Image Process. 18(12), 2706–2717 (2009)

    Article  MathSciNet  Google Scholar 

  30. H. Tan, X. Zeng, S. Lai, Y. Liu and M. Zhang, Joint demosaicing and denoising of noisy bayer images with ADMM, in 2017 IEEE International Conference on Image Processing (ICIP), 2017.

  31. F. Heide, M. Steinberger, Y.-T. Tsai, M. Rouf, D. Pajak, D. Reddy, O. Gallo, J. Liu, W. Heidrich, K. Egiazarian, J. Kautz and K. Pulli, Flexisp: a flexible camera image processing framework. ACM Trans. Graph, vol. 33, no. 6, 2014.

  32. Tang, R., Chen, L., Zhang, R., et al.: Medical image super-resolution with laplacian dense network. Multimed Tools Appl (2020). https://doi.org/10.1007/s11042-020-09845-y

    Article  Google Scholar 

  33. Din, S., Paul, A., Ahmad, A.: Smart embedded system based on demosaicking for enhancement of surveillance systems. Comput. Electr. Eng. 86, 106731 (2020)

    Article  Google Scholar 

  34. Liu, F., Lihui Chen, LuLu., Ahmad, A., Jeon, G., Yang, X.: Medical image fusion method by using Laplacian pyramid and convolutional sparse representation. Concurrency Computat Pract Exper. 32, e5632 (2020). https://doi.org/10.1002/cpe.5632

    Article  Google Scholar 

  35. Wei, S., Wei, Wu., Jeon, G., Ahmad, A., Yang, X.: Improving resolution of medical images with deep dense convolutional neural network. Concurrency Computat Pract Exper. 32, e5084 (2020). https://doi.org/10.1002/cpe.5084

    Article  Google Scholar 

  36. Wu, J., Anisetti, M., Wu, W., Damiani, E., Jeon, G.: Bayer Demosaicking With Polynomial Interpolation. IEEE Trans. Image Process. 25(11), 5369–5382 (2016). https://doi.org/10.1109/TIP.2016.2604489

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This project was funded by the Deanship of Scientific Research (DSR) at King Abdulaziz University, Jeddah, under grant no. D-1441-183-611. The authors, therefore, acknowledge with thanks DSR for technical and financial support.

Author information

Authors and Affiliations

  1. Department of Information Systems, Faculty of Computing and IT, King Abdulaziz University, Jeddah, Saudi Arabia

    Alaa O. Khadidos

  2. Department of Information Technology, Faculty of Computing and IT, King Abdulaziz University, Jeddah, Saudi Arabia

    Adil O. Khadidos, Fazal Qudus Khan & Georgios Tsaramirsis

  3. Department of Computer Science, Air University, Islamabad, 44000, Pakistan

    Awais Ahmad

Authors
  1. Alaa O. Khadidos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adil O. Khadidos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fazal Qudus Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Georgios Tsaramirsis
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Awais Ahmad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Awais Ahmad.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khadidos, A.O., Khadidos, A.O., Khan, F.Q. et al. Bayer image demosaicking and denoising based on specialized networks using deep learning. Multimedia Systems 27, 807–819 (2021). https://doi.org/10.1007/s00530-020-00707-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-020-00707-z

Keywords

Search

Navigation