Skip to main content
Springer Nature Link
Log in

Iterative spectral correlation based multispectral image demosaicking

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

Multispectral imaging systems with a multispectral filter array (MSFA) provide an affordable and portable way to capture multispectral images (MSIs) that have a variety of applications in different fields. These systems generate raw images initially and require an effective multispectral image demosaicking technique for reconstructing the MSI from the raw data. These demosaicking methods require proper usage of the spectral correlation available between the bands of MSI to generate high quality MSI. However, the existing demosaicking methods only partially utilize this spectral correlation, as they use spectral correlation only in the initial estimation of MSI. In this work, we utilize the spectral correlation between bands iteratively and finally enhance the quality of the generated image using median filtering based image enhancement. The exploratory results on the two standard datasets publicise the quality of the presented method on the various metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data availability

No datasets were generated or analysed during the current study.

References

  1. Chen, I., Lin, H.: Detection, counting and maturity assessment of cherry tomatoes using multi-spectral images and machine learning techniques. In: Proceedings of International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications( VISIGRAPP), pp. 759–766 (2020)

  2. Martinez-Herrera, S.E., Benezeth, Y., Boffety, M., Emile, J.-F., Marzani, F., Lamarque, D., Goudail, F.: Identification of precancerous lesions by multispectral gastroendoscopy. SIViP 10, 455–462 (2016)

    Article  Google Scholar 

  3. Lu, G., Fei, B.: Medical hyperspectral imaging: a review. J. Biomed. Opt. 19(1), 010901 (2014)

    Article  Google Scholar 

  4. Zang, Y., Fu, C., Yang, D., Li, H., Ding, C., Liu, Q.: Transformer fusion and histogram layer multispectral pedestrian detection network. SIViP 17(7), 3545–3553 (2023)

    Article  Google Scholar 

  5. Mangai, U.G., Samanta, S., Das, S., Chowdhury, P.R., Varghese, K., Kalra, M.: A hierarchical multi-classifier framework for landform segmentation using multi-spectral satellite images-a case study over the Indian subcontinent. In: IEEE Fourth Pacific-Rim Symposium on Image and Video Technology, pp. 306–313 (2010)

  6. Qin, J., Chao, K., Kim, M.S., Lu, R., Burks, T.F.: Hyperspectral and multi spectral imaging for evaluating food safety and quality. J. Food Eng. 118(2), 157–171 (2013)

    Article  Google Scholar 

  7. Miao, L., Qi, H.: The design and evaluation of a generic method for generating mosaicked multispectral filter arrays. IEEE Trans. Image Process. 15(9), 2780–2791 (2006)

    Article  Google Scholar 

  8. Gupta, M., Rathi, V., Goyal, P.: Adaptive and progressive multispectral image demosaicking. IEEE Trans. Comput. Imaging 8, 69–80 (2022)

    Article  MathSciNet  Google Scholar 

  9. Menon, D., Menon, G.: Color image demosaicking: an overview. Signal Process. Image Commun. 26(8–9), 518–533 (2011)

    Article  Google Scholar 

  10. Li, X., Gunturk, B., Zhang, L.: Image demosaicing: a systematic survey. Proc. SPIE 6822, 1–15 (2008)

    Google Scholar 

  11. Kwan, C., Chou, B., Bell, J.F., III.: Comparison of deep learning and conventional demosaicing algorithms for mastcam images. Electronics 8(3), 308 (2019)

    Article  Google Scholar 

  12. Kwan, C., Chou, B.: A comparative study of conventional and deep learning approaches for demosaicing mastcam images. In: Signal Processing, Sensor/Information Fusion, and Target Recognition XXVIII, vol. 11018, pp. 332–339 (2019)

  13. Kwan, C.: Demosaicing mastcam images using a new color filter array. Signal Image Process. Int. J. (SIPIJ) 11(3) (2020) https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3640348

  14. Rathi, V., Goyal, P.: Convolution filter based efficient multispectral image demosaicking for compact msfas. In: Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications-Volume 4: VISAPP, pp. 112–121 (2021)

  15. Tsagkatakis, G., Bloemen, M., Geelen, B., Jayapala, M., Tsakalides, P.: Graph and rank regularized matrix recovery for snapshot spectral image demosaicing. IEEE Trans. Comput. Imaging 5(2), 301–316 (2019)

    Article  Google Scholar 

  16. Brauers, J., Aach, T.: A color filter array based multispectral camera. In: 12. Workshop Farbbildverarbeitung, pp. 55–64 (2006)

  17. Rathi, V., Gupta, M., Goyal, P.: A new generic progressive approach based on spectral difference for single-sensor multispectral imaging system. In: Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications-Volume 4: VISAPP, pp. 329–336 (2021)

  18. Miao, L., Ramanath, R., Snyder, W.E.: Binary tree-based generic demosaicking algorithm for multispectral filter arrays. IEEE Trans. Image Process. 15(11), 3550–3558 (2006)

    Article  Google Scholar 

  19. Gupta, M., Ram, M.: Weighted bilinear interpolation based generic multispectral image demosaicking method. J. Graph. Era Univ. 7(2), 108–118 (2019)

    Google Scholar 

  20. Rathi, V., Goyal, P.: Generic multispectral image demosaicking algorithm and new performance evaluation metric. In: Computer Vision and Image Processing, pp. 45–57 (2022)

  21. Monno, Y., Tanaka, M., Okutomi, M.: Multispectral demosaicking using adaptive kernel upsampling. In: Proceedings of IEEE International Conference on Image Processing, pp. 3157–3160 (2011)

  22. Monno, Y., Tanaka, M., Okutomi, M.: Multispectral demosaicking using guided filter. In: Proceedings of the SPIE Electronic Imaging Annual Symposium, pp. 8299, 82990O (2012) https://www.spiedigitallibrary.org/conference-proceedings-of-spie/8299/1/Multispectral-demosaicking-using-guided-filter/10.1117/12.906168.short#_=_

  23. Monno, Y., Kiku, D., Kikuchi, S., Tanaka, M., Okutomi, M.: Multispectral demosaicking with novel guide image generation and residual interpolation. In: Proceedings of IEEE International Conference on Image Processing, pp. 645–649 (2014)

  24. Monno, Y., Kikuchi, S., Tanaka, M., Okutomi, M.: A practical one-shot multispectral imaging system using a single image sensor. IEEE Trans. Image Process. 24(10), 3048–3059 (2015)

    Article  MathSciNet  Google Scholar 

  25. Mihoubi, S., Losson, O., Mathon, B., Macaire, L.: Multispectral demosaicking using intensity-based spectral correlation. In: Proceedings of the 5th International Conference on Image Processing Theory, Tools and Applications, pp. 461–466 (2015)

  26. Mihoubi, S., Losson, O., Mathon, B., Macaire, L.: Multispectral demosaicing using pseudo-panchromatic image. IEEE Trans. Comput. Imaging 3(4), 982–995 (2017)

    Article  MathSciNet  Google Scholar 

  27. Sun, B., Zhao, Z., Xie, D., Yuan, N., Yu, Z., Chen, F., Cao, C., Dravo, V.W.: Sparse spectral signal reconstruction for one proposed nine-band multispectral imaging system. Mech. Syst. Signal Process. 141, 106627 (2020)

    Article  Google Scholar 

  28. Gupta, M.: Generalizing spectral difference method for multispectral image demosaicking and analyzing the role of MSFA patterns. In: 2020 9th International Conference System Modeling and Advancement in Research Trends (SMART), pp. 416–420 (2020)

  29. Mizutani, J., Ogawa, S., Shinoda, K., Hasegawa, M., Kato, S.: Multispectral demosaicking algorithm based on inter-channel correlation. In: Proceedings of the IEEE Visual Communications and Image Processing Conference, pp. 474–477 (2014)

  30. Shopovska, I., Jovanov, L., Philips, W.: Rgb-nir demosaicing using deep residual u-net. In: 26th Telecommunications Forum, pp. 1–4 (2018)

  31. Habtegebrial, T.A., Reis, G., Stricker, D.: Deep convolutional networks for snapshot hypercpectral demosaicking. In: 10th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing, pp. 1–5 (2019)

  32. Dijkstra, K., Loosdrecht, J., Schomaker, L.R.B., Wiering, M.A.: Hyperspectral demosaicking and crosstalk correction using deep learning. Mach. Vis. Appl. 30(1), 1–21 (2019)

    Article  Google Scholar 

  33. Shinoda, K., Yoshiba, S., Hasegawa, M.: Deep demosaicking for multispectral filter arrays. arXiv:1808.08021 (2018)

  34. Feng, K., Zhao, Y., Chan, J.C.-W., Kong, S.G., Zhang, X., Wang, B.: Mosaic convolution-attention network for demosaicing multispectral filter array images. IEEE Trans. Comput. Imaging 7, 864–878 (2021)

    Article  Google Scholar 

  35. Niu, Y., Ouyang, J., Zuo, W., Wang, F.: Low cost edge sensing for high quality demosaicking. IEEE Trans. Image Process. 28(5), 2415–2427 (2019)

    Article  MathSciNet  Google Scholar 

  36. Freeman, T.W.: Median filter for reconstructing missing color samples. United States Patent, No. 4724395 (1988)

  37. Yasuma, F., Mitsunaga, T., Iso, D., Nayar, S.K.: Generalized assorted pixel camera: postcapture control of resolution, dynamic range, and spectrum. IEEE Trans. Image Process. 19(9), 2241–2253 (2010)

  38. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  39. Low Resolution Kodak Image Dataset. http://r0k.us/graphics/kodak/. Accessed 18 June 2024

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, National Institute of Technology Kurukshetra, Kurukshetra, Haryana, India

    Vishwas Rathi

  2. Computer Science and Engineering and Information Technology, Jaypee University of Information Technology, Solan, Himachal Pardesh, India

    Kapil Rana

  3. Department of Computer Science and Engineering, Indian Institute of Technology Ropar, Rupnagar, Punjab, India

    Kapil Rana & Puneet Goyal

  4. NIMS Institute of Engineering and Technology, NIMS University, Jaipur, Rajasthan, India

    Puneet Goyal

Authors
  1. Vishwas Rathi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kapil Rana
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Puneet Goyal
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

VR and KR wrote the manuscript. VR and PG did the conceptualization of the method. VR made the figures and written algorithm. KR generated the tables. All authors reviewed the manuscript as last.

Corresponding author

Correspondence to Vishwas Rathi.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A The number of iterations between band i and band j

Here, we show the number of iterations between band i and band j for 11–16 bands multispectral images in Tables 10, 11, 12, 13, 14 and 15.

Fig. 4
figure 4

Performance of the proposed method on 11-band MSIs from the CAVE dataset across varying iterations. The red triangle indicates the performance reported in the manuscript. itr denotes the iterations used in the manuscript as mentioned in Table 10, ‘\(itr + x\)’, \( (x = {1, 2, 3, 4, 5})\) represents iterations increased by x, and \(itr*\) indicates the single iteration

Full size image
Fig. 5
figure 5

Performance of the proposed method on 12-band MSIs from the CAVE dataset across varying iterations. The red triangle indicates the performance reported in the manuscript. itr denotes the iterations used in the manuscript as mentioned in Table 11, ‘\(itr + x\)’, \( (x = {1, 2, 3, 4, 5})\) represents iterations increased by x, and \(itr*\) indicates the single iteration

Full size image
Fig. 6
figure 6

Performance of the proposed method on 16-band MSIs from the CAVE dataset across varying iterations. The red triangle indicates the performance reported in the manuscript. itr denotes the iterations used in the manuscript as mentioned in Table 15, ‘\(itr + x\)’, \( (x = {-2, -1, 0, 1, 2, 3, 4, 5})\) represents iterations increased by x, and \(itr*\) indicates the single iteration

Full size image

Appendix B Impact of number of iterations on the proposed method

Here, we show the impact of the number of iterations on the proposed method. We run the proposed method for greater or fewer iterations than the one mentioned in the manuscript and observe that the performance of the proposed algorithm has decreased. Below, we show the performance of the proposed method on 11-band, 12-band and 16-band multispectral images on the CAVE dataset. In the following figures, the number of iterations is increased or decreased sequentially (Figs. 4, 5, 6).

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rathi, V., Rana, K. & Goyal, P. Iterative spectral correlation based multispectral image demosaicking. SIViP 18, 7873–7886 (2024). https://doi.org/10.1007/s11760-024-03435-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-024-03435-3

Keywords