Skip to main content
SpringerLink
Log in

Low-dimensional manifold model for demosaicking from a RGBW color filter array

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

In this article, we introduce a variational demosaicking model that restores full-color images from sampled data acquired with a RGBW color filter array (CFA). The proposed model employs the concept of the low-dimensional patch manifold model (LDMM) in Shi et al. (J Sci Comput 75(2):638–656, 2018) and inter-channel correlation terms. The LDMM enables regions without color data to be filled out smoothly from given sparse data, while conserving textures. Moreover, the inter-correlation terms defined in the gradient domain help diminish color artifacts in demosaicked images. We also present an efficient iterative algorithm for solving the proposed model. Numerical experiments validate the effectiveness of our model for demosaicking images acquired with RGBW CFAs as compared to the state-of-the-art models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. http://r0k.us/graphics/kodak/.

References

  1. Bayer, B.E.: Color imaging array. U.S. Patent 3, 971, 065 (1976)

  2. Bentley, J.L.: Multidimensional binary search trees used for associative searching. Commun. ACM 18(9), 509–517 (1975)

    Article  MathSciNet  Google Scholar 

  3. Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imaging Sci. 3(3), 492–526 (2010)

    Article  MathSciNet  Google Scholar 

  4. Buades, A., Coll, B., Morel, J., Sbert, C.: Non local demosaicing. In: CMLA (2007)

  5. Buades, A., Coll, B., Morel, J.M., Sbert, C.: Self-similarity driven color demosaicking. IEEE Trans. Image Process. 18(6), 1192–1202 (2009)

    Article  MathSciNet  Google Scholar 

  6. Carlsson, G., Ishkhanov, T., De Silva, V., Zomorodian, A.: On the local behavior of spaces of natural images. Int. J. Comput. Vis. 76(1), 1–12 (2008)

    Article  MathSciNet  Google Scholar 

  7. Chang, K., Ding, P.L.K., Li, B.: Color image demosaicking using inter-channel correlation and nonlocal self-similarity. Signal Process. Image Commun. 39, 264–279 (2015)

    Article  Google Scholar 

  8. Condat, L.: A generic variational approach for demosaicking from an arbitrary color filter array. In: 2009 16th IEEE International Conference on Image Processing (ICIP), pp. 1625–1628 (2009)

  9. Condat, L.: Color filter array design using random patterns with blue noise chromatic spectra. Image Vis. Comput. 28(8), 1196–1202 (2010)

    Article  Google Scholar 

  10. Condat, L., Mosaddegh, S.: Joint demosaicking and denoising by total variation minimization. In: Proceedings of the 19th IEEE International Conference on Image Processing, pp. 2781–2784 (2012)

  11. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)

    Article  MathSciNet  Google Scholar 

  12. Duran, J., Buades, A.: Self-similarity and spectral correlation adaptive algorithm for color demosaicking. IEEE Trans. Image Process. 23(9), 4031–4040 (2014)

    Article  MathSciNet  Google Scholar 

  13. Esser, E., Zhang, X., Chan, T.F.: A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science. SIAM J. Imaging Sci. 3(4), 1015–1046 (2010)

    Article  MathSciNet  Google Scholar 

  14. Farsiu, S., Elad, M., Milanfar, P.: Multiframe demosaicing and super-resolution of color images. IEEE Trans. Image Process. 15(1), 141–159 (2006)

    Article  Google Scholar 

  15. Gao, D., Wu, X., Shi, G., Zhang, L.: Color demosaicking with an image formation model and adaptive PCA. J. Vis. Commun. Image Represent. 23(7), 1019–1030 (2012)

    Article  Google Scholar 

  16. Gilboa, G., Osher, S.: Nonlocal operators with applications to image processing. Multiscale Model. Simul. 7(3), 1005–1028 (2008)

    Article  MathSciNet  Google Scholar 

  17. Gindele, E., Gallagher, A.: Sparsely sampled image sensing device with color and luminance photosites. U.S. Patent 6,476,865 (2002)

  18. Goldstein, T., Osher, S.: The split Bregman method for L1-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)

    Article  MathSciNet  Google Scholar 

  19. Gunturk, B.K., Glotzbach, J., Altunbasak, Y., Schafer, R.W., Mersereau, R.M.: Demosaicking: color filter array interpolation. IEEE Signal Process. Mag. 22(1), 44–54 (2005)

    Article  Google Scholar 

  20. Hamilton Jr, J.F.: Adaptive color plan interpolation in signal sensor color electronic camera. U.S. Patent 5, 629, 734 (1997)

  21. Kang, M., Kang, M., Jung, M.: Image colorization based on a generalization of the low dimensional manifold model. J. Sci. Comput. 77(2), 911–935 (2018)

    Article  MathSciNet  Google Scholar 

  22. Keren, D., Osadchy, M.: Restoring subsampled color images. Mach. Vis. Appl. 11(4), 197–202 (1999)

    Article  Google Scholar 

  23. Kijima, T., Nakamura, H., Compton, J.T., Hamilton Jr, J.F., DeWeese, T.E.: Image sensor with improved light sensitivity. U.S. Patent 7, 916, 362 (2011)

  24. Kumar, M., Morales, E., Adams, J., Hao, W.: New digital camera sensor architecture for low light imaging. In: 16th IEEE International Conference on Image Processing, pp. 2681–2684 (2009)

  25. Laroche, C.A., Prescott, M.A.: Apparatus and method for adaptively interpolating a full color image utilizing chrominance gradients. U.S. Patent 5, 373, 322 (1994)

  26. Lukac, R., Plataniotis, K.N.: Color filter arrays: design and performance analysis. IEEE Trans. Consum. Electron. 51(4), 1260–1267 (2005)

    Article  Google Scholar 

  27. Mairal, J., Elad, M., Sapiro, G.: Sparse representation for color image restoration. IEEE Trans. Image Process. 17(1), 53–59 (2008)

    Article  MathSciNet  Google Scholar 

  28. Menon, D., Andriani, S., Calvagno, G.: Demosaicing with directional filtering and a posteriori decision. IEEE Trans. Image Process. 16(1), 132–141 (2007)

    Article  MathSciNet  Google Scholar 

  29. Menon, D., Calvagno, G.: Regularization approaches to demosaicking. IEEE Trans. Image Process. 18(10), 2209–2220 (2009)

    Article  MathSciNet  Google Scholar 

  30. Osher, S., Shi, Z., Zhu, W.: Low dimensional manifold model for image processing. SIAM J. Imaging Sci. 10(4), 1669–1690 (2017)

    Article  MathSciNet  Google Scholar 

  31. Peyré, G.: Manifold models for signals and images. Comput. Vis. Image Underst. 113(2), 249–260 (2009)

    Article  Google Scholar 

  32. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1–4), 259–268 (1992)

    Article  MathSciNet  Google Scholar 

  33. Saito, T., Komatsu, T.: Demosaicing approach based on extended color total-variation regularization. In: 15th IEEE International Conference on Image Processing, pp. 885–888 (2008)

  34. Shi, Z., Osher, S., Zhu, W.: Weighted nonlocal laplacian on interpolation from sparse data. J. Sci. Comput. 73(2–3), 1164–1177 (2017)

    Article  MathSciNet  Google Scholar 

  35. Shi, Z., Osher, S., Zhu, W.: Generalization of the weighted nonlocal Laplacian in low dimensional manifold model. J. Sci. Comput. 75(2), 638–656 (2018)

    Article  MathSciNet  Google Scholar 

  36. Tachi, M.: Image processing device, image processing method, and program pertaining to image correction. U.S. Patent 8, 314, 863 (2012)

  37. Wang, J., Zhang, C., Hao, P.: New color filter arrays of high light sensitivity and high demosaicking performance. In: Proceedings of the 18th IEEE International Conference on Image Processing, pp. 3153–3156 (2011)

  38. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  39. Yamagami, T., Sasaki, T., Suga, A.: Image signal processing apparatus having a color filter with offset luminance filter elements. U.S. Patent 5, 323, 233 (1994)

Download references

Acknowledgements

Myeongmin Kang was supported by the NRF (2018R1C1B4A01019631). Miyoun Jung was supported by Hankuk University of Foreign Studies Research Fund and the NRF (2017R1A2B1005363).

Author information

Authors and Affiliations

  1. Department of Mathematics, Chungnam National University, Daejeon, Korea

    Myeongmin Kang

  2. Department of Mathematics, Hankuk University of Foreign Studies, Yongin, Korea

    Miyoun Jung

Authors
  1. Myeongmin Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Miyoun Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Miyoun Jung.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 427 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kang, M., Jung, M. Low-dimensional manifold model for demosaicking from a RGBW color filter array. SIViP 14, 143–150 (2020). https://doi.org/10.1007/s11760-019-01535-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-019-01535-z

Keywords

Search

Navigation