Skip to main content
SpringerLink
Log in

Reconstruction of missing color-channel data using a three-step back propagation neural network

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Demosaicking aims to approximate missing color pixels through analysis of the geometric structure between given color pixels and missing color pixels. In this paper, we introduce an efficient adaptive demosaicking method based on back propagation (BP) neural network (BP-NN). We firstly reconstruct the green channel using one BPNN, and then refine the green channel utilizing another BPNN based on the color difference. With the whole green channel interpolated, we reconstruct the red/blue channel using the color difference between the green channel and red/blue channel in a local region. Finally, we refine the red/blue channel using the third BPNN. Regarding the interpolation issue, different image features have completely different properties, such as smooth regions, edges, and textures. Consequently, it is necessary to identify an adaptive model to estimate the relation among neighboring color pixels. We provide the adaptive BP-NN based demosaicking algorithm which can reduce blurring through recovery of missing pixels by a learning process, and also use a pre-trained fixed network to reduce computational complexity. Experimental results demonstrate that the proposed method outperforms extant approaches in PSNR, computational complexity, and visual quality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Lukac R, Plataniotis KN, Hatzinakos D (2005) Color image zooming on the Bayer pattern. IEEE Trans Circuits Syst Video Technol 15(11):1475–1492

    Article  Google Scholar 

  2. Gunturk BK, Glotzbach J, Altunbask Y, Schafer RW, Mersereau RM (2005) Demosaicing: color filter array interpolation. IEEE Signal Process Mag 22(1):44–54

    Article  Google Scholar 

  3. Hamilton JF, Adams JE (1997) Adaptive color plane interpolation in single sensor color electronic camera. US Patent 5 629 734

  4. Pei S-C, Tam I-K (2003) Effective color interpolation in CCD color filter arrays using signal correlation. IEEE Trans Circuits Syst Video Technol 13(6):503–513

    Article  Google Scholar 

  5. Chang L, Tam Y-P (2004) Effective use of spatial and spectral correlations for color filter array demosaicing. IEEE Trans Consum Electron 50(1):355–365

    Article  Google Scholar 

  6. Hirakawa K, Parks TW (2005) Adaptive homogeneity-directed demosaicing algorithm. IEEE Trans Image Process 14(3):360–369

    Article  Google Scholar 

  7. Lian NX, Chang L, Tan YP, Zagorodnov V (2007) Adaptive filtering for color filter array demosaicking. IEEE Trans Image Process 16(10):2515–2525

    Article  MathSciNet  Google Scholar 

  8. Chung K-H, Chan Y-H (2006) Color demosaicing using variance of color differences. IEEE Trans Image Process 15(10):2944–2955

    Article  Google Scholar 

  9. Menon D, Andriani S, Calvagno G (2007) Demosaicing with directional filtering and a posteriori decision. IEEE Trans Image Process 16(1):132–141

    Article  MathSciNet  Google Scholar 

  10. Dengwen Z, Xiaoliu X, Weiming D (2012) Colour demosaicking with directional filtering and weighting. IET Image Process 6(8):1084–1092

    Article  MathSciNet  Google Scholar 

  11. Li JSJ, Randhawa S (2009) Color filter array demosaicking using high-order interpolation techniques with a weighted median filter for sharp color edge preservation. IEEE Trans Image Process 18(9):1946–1957

    Article  MathSciNet  Google Scholar 

  12. Su C-Y, Kao W-C (2009) Effective demosaicing using subband correlation. IEEE Trans Consum Electron 55(1):199–204

    Article  Google Scholar 

  13. Chen W-J, Chang P-Y (2012) Effective demosaicking algorithm based on edge property for color filter arrays. Digit Signal Process 22(1):163–169

    Article  MathSciNet  Google Scholar 

  14. Pekkucuksen I, Altunbasak Y (2012) Edge strength filter-based color filter array interpolation. IEEE Trans Image Process 21(1):393–397

    Article  MathSciNet  Google Scholar 

  15. Pekkucuksen I, Altunbasak Y (2013) Multiscale gradients-based color filter array interpolation. IEEE Trans Image Process 22(1):157–165

    Article  MathSciNet  Google Scholar 

  16. Kim J, Jeon G, Jeong J (2014) Demosaicking using geometric duality and dilated directional differentiation. Optics Commun 324:194–201

    Article  Google Scholar 

  17. Menon D, Calvagno G (2009) Regularization approaches to demosaicking. IEEE Trans Image Process 18(10):2209–2220

    Article  MathSciNet  Google Scholar 

  18. Dubois E (2005) Frequency-domain methods for demosaicking of Bayer-sampled color images. IEEE Signal Process Lett 12(12):847–850

    Article  Google Scholar 

  19. Leung B, Jeon G, Dubois E (2011) Least-squares luma-chroma demultiplexing algorithm for Bayer demosaicking. IEEE Trans Image Process 20(7):1885–1894

    Article  MathSciNet  Google Scholar 

  20. Jeon G, Dubois E (2013) Demosaicking of noisy Bayer-sampled color images with least-squares luma-chroma demultiplexing and noise level estimation. IEEE Trans Image Process 22(1):146–156

    Article  MathSciNet  Google Scholar 

  21. Chen X, Jeon G, Jeong J, He L (2015) Multidirectional weighted interpolation and refinement method for Bayer pattern CFA demosaicking. IEEE Trans Circuits Syst Video Technol 25(8):1271–1282

    Article  Google Scholar 

  22. Wang L, Jeon G (2015) Bayer pattern CFA demosaicking based on multi-directional weighted interpolation and guided filter. IEEE Signal Process Lett 22(11):2083–2087

    Article  Google Scholar 

  23. Wu J, Anisetti M, Wu W, Damiani E, Jeon G (2016) Bayer demosaicking with polynomial interpolation. IEEE Trans Image Process 25(11):5369–5382

    Article  MathSciNet  Google Scholar 

  24. Wang J, Wu J, Wu Z, Jeon G, Jeong J (2017) Bilateral filtering and directional differentiation for Bayer demosaicking. IEEE Sens J 17(3):726–734

    Article  Google Scholar 

  25. Wang J, Wu J, Wu Z, Jeon G (2016) Taylor series and adaptive directional selection for real time demosaicking. Displays 45:14–25

    Article  Google Scholar 

  26. Seo G, Choi H, Lee C (2009) Efficient implementation of neural network deinterlacing. In: Proc. SPIE, vol 7245, pp 724519

  27. Ding S, Su C, Yu J (2011) An optimizing BP neural network algorithm based on genetic algorithm. J Artif Intell Rev 36(2):153–162

    Article  Google Scholar 

Download references

Acknowledgment

This paper is sponsored by National Natural Science Foundation of China (No. 61501359, 61771378), by the Framework of International Cooperation Program managed by the NRF of Korea under Grant NRF-2016K1A3A1A25003543 and by the “Ministero degli Affari Esteri e della Cooperazione Internazionale” of Italy under Grant PGR00217.

Author information

Authors and Affiliations

  1. School of Physics and Optoelectronic Engineering, Xidian University, Xi’an, Shaanxi, China

    Jin Wang

  2. Dipartimento di Informatica, Università degli Studi di Milano, Crema, Italy

    Marco Anisetti

  3. School of Electronic Engineering, Xidian University, Xi’an, Shaanxi, China

    Gwanggil Jeon

  4. Department of Embedded Systems Engineering, Incheon National University, Incheon, Korea

    Gwanggil Jeon

Authors
  1. Jin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marco Anisetti
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gwanggil Jeon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gwanggil Jeon.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, J., Anisetti, M. & Jeon, G. Reconstruction of missing color-channel data using a three-step back propagation neural network. Int. J. Mach. Learn. & Cyber. 10, 2631–2642 (2019). https://doi.org/10.1007/s13042-018-0850-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-018-0850-5

Keywords

Search

Navigation