Skip to main content
SpringerLink
Log in

Multispectral Image Quality Improvement Based on Global Iterative Fusion Constrained by Meteorological Factors

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

It has been proven that the refractive index is related to meteorological parameters in physics. The temperature changes the atmospheric and lens refractive indices, resulting in image degradation. Image restoration aims to recover the sharp image from the degraded images. It is also the basis of many computer vision tasks. A series of methods have been explored and used in this area. Sometimes, meteorological factors cause image degradation. Most of the existing image restoration methods do not consider meteorological factors’ influence on image degradation. How meteorological factors affect image quality is not yet known. A multispectral image dataset with corresponding meteorological parameters is presented to solve the problem. We propose a novel multispectral image restoration algorithm using global iterative fusion. The proposed method firstly enhances image edge features through spatial filtering. Then, the Gaussian function is used to constrain the weights between each channel in the image. Finally, a global iterative fusion method is used to fuse image spatial and spectral features to obtain an improved multispectral image. The algorithm explores the impact of meteorological factors on image quality. It considers the impact of atmospheric factors on image quality and incorporates it into the image restoration process. Extensive experimental results illustrate that the method achieves favorable performance on real data. The proposed algorithm is also more robust than other state-of-the-art algorithms. In this paper, we present an algorithm for improving the quality of multispectral images. The proposed algorithm incorporates the influence of meteorological parameters into the image restoration method to better describe the relationship between different spectral channels. Extensive experiments are conducted to validate the effectiveness of the algorithm. Additionally, we investigate the impact of near-surface meteorological parameters on multispectral image quality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Algorithm 2
Fig. 5
Algorithm 3
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data Availibility

All data included in this study are available upon request by contact with the corresponding author.

References

  1. Rucco A, Sujit P, Aguiar AP, De Sousa JB, Pereira FL. Optimal rendezvous trajectory for unmanned aerial-ground vehicles. IEEE Trans Aerosp Electron Syst. 2017;54(2):834–47. https://doi.org/10.1109/TAES.2017.2767958.

    Article  Google Scholar 

  2. Kimura R, Yoshimura Y. The contribution of low contrast-preferring neurons to information representation in the primary visual cortex after learning. Sci Adv. 2021;7(48):9976. https://doi.org/10.1126/sciadv.abj9976.

    Article  Google Scholar 

  3. Wu S, Huang J, Feng Y, Sun B. Multiple reliable structured patches for object tracking. Cogn Comput. 2021;13(4):1593–602. https://doi.org/10.1007/s12559-020-09741-5.

    Article  Google Scholar 

  4. Hartley R, Kamgar-Parsi B, Narber C. Using roads for autonomous air vehicle guidance. IEEE Trans Intell Transp Syst. 2018;19(12):3840–9. https://doi.org/10.1109/TITS.2018.2799485.

    Article  Google Scholar 

  5. Yang W, Wang S, Fang Y, Wang Y, Liu J. From fidelity to perceptual quality: a semi-supervised approach for low-light image enhancement. In: 2020 IEEE Conference on Computer Vision and Pattern Recognition(CVPR); 2020. p. 3063–3072. https://doi.org/10.1109/CVPR42600.2020.00313.

  6. Ren W, Liu S, Ma L, Xu Q, Xu X, Cao X, Du J, Yang MH. Low-light image enhancement via a deep hybrid network. IEEE Trans Image Process. 2019;28(9):4364–75. https://doi.org/10.1109/TIP.2019.2910412.

    Article  MathSciNet  Google Scholar 

  7. Wang R, Zhang Q, Fu CW, Shen X, Jia J. Underexposed photo enhancement using deep illumination estimation. In: 2019 IEEE Conference on Computer Vision and Pattern Recognition(CVPR); 2019. p. 6849–6857. https://doi.org/10.1109/CVPR.2019.00701.

  8. Shen Z, Wang W, Lu X, Shen J, Ling H, Xu T, Shao L. Human-aware motion deblurring. In: 2019 IEEE International Conference on Computer Vision(ICCV); 2019. p. 5572–5581. https://doi.org/10.1109/ICCV.2019.00567.

  9. Shen Z, Lai W-S, Xu T, Kautz J, Yang M-H. Deep semantic face deblurring. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition(CVPR); 2018. p. 8260–8269. https://doi.org/10.1109/CVPR.2018.00862.

  10. Kupyn O, Budzan V, Mykhailych M, Mishkin D, Matas J. Deblurgan: blind motion deblurring using conditional adversarial network. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition(CVPR); 2018. p. 8183–8192. https://doi.org/10.1109/CVPR.2018.00854.

  11. Chen S-J, Shen H-L. Multispectral image out-of-focus deblurring using interchannel correlation. IEEE Trans Image Process. 2015;24(11):4433–45. https://doi.org/10.1109/TIP.2015.2465162.

    Article  MathSciNet  Google Scholar 

  12. Iqbal M, Riaz MM, Ghafoor A, Ahmad A, Ali SS. Out of focus multi-spectral image de-blurring using texture extraction and modified Fourier transform. Multimed Tools Appl. 2021;80(8):12671–84. https://doi.org/10.1007/s11042-020-10232-w.

    Article  Google Scholar 

  13. Li X, Wu J, Liu Z, Zha H. Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: 2018 European Conference on Computer Vision(ECCV); 2018. p. 254–269. https://doi.org/10.1007/978-3-030-01234-2_16.

  14. He Z, Vishal MP. Densely connected pyramid dehazing network. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition(CVPR); 2018. p. 3194–3203. https://doi.org/10.1109/CVPR.2018.00337.

  15. Ren W, Ma L, Zhang J, Pan J, Cao X, Liu W, Yang MH. Gated fusion network for single image dehazing. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition(CVPR); 2018. p. 3253–3261. https://doi.org/10.1109/CVPR.2018.00343.

  16. Li H, Li G, Yan W, He G, Lin L. Synergy effect and its application in led-multispectral imaging for improving image quality. Opt Commun. 2019;438(1):6–12. https://doi.org/10.1016/j.optcom.2018.12.091.

    Article  Google Scholar 

  17. Xie Q, Zhao Q, Meng D, Xu Z. Kronecker-basis-representation based tensor sparsity and its applications to tensor recovery. IEEE Trans Pattern Anal Mach Intell. 2017;54(8):1888–902. https://doi.org/10.1109/TPAMI.2017.2734888.

    Article  Google Scholar 

  18. He W, Yao Q, Li C, Yokoya N, Zhao Q, Zhang H, Zhang L. Non-local meets global: an iterative paradigm for hyperspectral image restoration. IEEE Trans Pattern Anal Mach Intell. 2022;44(4):2089–107. https://doi.org/10.1109/TPAMI.2020.3027563.

    Article  Google Scholar 

  19. Chen Y, Guo Y, Wang Y, Wang D, Peng C, He G. Denoising of hyperspectral images using nonconvex low rank matrix approximation. IEEE Trans Geosci Remote Sens. 2017;55(9):5366–80. https://doi.org/10.1109/TGRS.2017.2706326.

    Article  Google Scholar 

  20. Sun H, Zheng X, Lu X, Wu S. Spectral-spatial attention network for hyperspectral image classification. IEEE Trans Geosci Remote Sens. 2019;58(5):3232–45. https://doi.org/10.1109/TGRS.2019.2951160.

    Article  Google Scholar 

  21. Chen W, Zheng X, Lu X. Semisupervised spectral degradation constrained network for spectral super-resolution. IEEE Geosci Remote Sens Lett. 2021;19(1):1–5. https://doi.org/10.1109/LGRS.2021.3079961.

    Article  Google Scholar 

  22. Sun H, Zheng X, Lu X. A supervised segmentation network for hyperspectral image classification. IEEE Trans Image Process. 2021;30(1):2810–25. https://doi.org/10.1109/TIP.2021.3055613.

    Article  Google Scholar 

  23. Malik M, Majumder S. Weather-predicting atmospheric modulation transfer function. In: 2013 3rd IEEE International Advance Computing Conference(IACC); 2013. p. 1613–1619. https://doi.org/10.1109/IAdCC.2013.6514469.

  24. Saiga R, Takeuchi A, Uesugi K, Terada Y, Suzuki Y, Mizutani R. Method for estimating modulation transfer function from sample images. Micron. 2018;105:64–9. https://doi.org/10.1016/j.micron.2017.11.009.

    Article  Google Scholar 

  25. Ren W, Zhang J, Ma L, Pan J, Cao X, Zuo W, Liu W, Yang M-H. Deep non-blind deconvolution via generalized low-rank approximation. Adv Neural Inf Process Syst (NeurIPS). 2018;31(1):297–307.

  26. Zhang M, Young GS, Tie Y, Gu X, Xu X. A new framework of designing iterative techniques for image deblurring. Pattern Recogn. 2022;124:108463. https://doi.org/10.1016/j.patcog.2021.108463.

    Article  Google Scholar 

  27. He K, Sun J, Tang X. Single image haze removal using dark channel prior. IEEE Trans Pattern Anal Mach Intell. 2010;33(12):2341–53. https://doi.org/10.1109/TPAMI.2010.168.

    Article  Google Scholar 

  28. Berman D, Avidan S, et al. Non-local image dehazing. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR); 2016. p. 1647–1682. https://doi.org/10.1109/CVPR.2016.185.

  29. Zhang Z, Zheng L, Piao Y, Tao S, Xu W, Gao T, Wu X. Blind remote sensing image deblurring using local binary pattern prior. Remote Sens. 2022;14(5):1276. https://doi.org/10.3390/rs14051276.

    Article  Google Scholar 

  30. Pan L, Hartley R, Liu M, Dai Y. Phase-only image based kernel estimation for single image blind deblurring. In: 2019 IEEE Conference on Computer Vision and Pattern Recognition(CVPR); 2019. p. 6034–6043. https://doi.org/10.1109/CVPR.2019.00619.

  31. Wang Z, Ren J, Zhang J, Luo P. Image deblurring aided by low-resolution events. Electronics. 2022;11(4):631. https://doi.org/10.3390/electronics11040631.

    Article  Google Scholar 

  32. Chen L, Fang F, Wang T, Zhang G. Blind image deblurring with local maximum gradient prior. In: 2019 IEEE Conference on Computer Vision and Pattern Recognition(CVPR); 2019. p. 1742–1750. https://doi.org/10.1109/CVPR.2019.00184.

  33. Shi Y, Wang N, Yang F, Zhang G, Li S, Liu X. Multispectral images deblurring via interchannel correlation relationship. In: 2021 4th International Conference on Information Communication and Signal Processing (ICICSP); 2021. p. 458–462. https://doi.org/10.1109/ICICSP54369.2021.9611913.

  34. Pan J, Hu Z, Su Z, Lee H-Y, Yang M-H. Soft-segmentation guided object motion deblurring. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR); 2016. p. 459–468. https://doi.org/10.1109/CVPR.2016.56.

  35. Yan Y, Ren W, Guo Y, Rui W, Cao X. Image deblurring via extreme channels prior. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR); 2017. p. 4003–4011. https://doi.org/10.1109/CVPR.2017.738.

  36. Bai Y, Cheung G, Liu X, Gao W. Graph-based blind image deblurring from a single photograph. IEEE Trans Image Process. 2019;28(3):1404–18. https://doi.org/10.1109/TIP.2018.2874290.

    Article  MathSciNet  Google Scholar 

  37. Wen F, Ying R, Liu Y, Liu P, Truong TK. A simple local minimal intensity prior and an improved algorithm for blind image deblurring. IEEE Trans Circuits Syst Video Technol. 2021;31(8):2923–37. https://doi.org/10.1109/TCSVT.2020.3034137.

    Article  Google Scholar 

  38. Wei X-X, Zhang L, Huang H. High-quality blind defocus deblurring of multispectral images with optics and gradient prior. Opt Express. 2020;28(7):106883–10704.

    Article  Google Scholar 

  39. Guo X, Yang Y, Wang C, Ma J. Image dehazing via enhancement, restoration, and fusion: a survey. Inf Fusion. 2022;86:146–70. https://doi.org/10.1016/j.inffus.2022.07.005.

    Article  Google Scholar 

  40. Butrimas S, Driggers RG, Holst GC, Kopeika NS, Zilberman A. Effects of aerosol modulation transfer function on target identification. Opt Eng. 2020;59(7):1–14. https://doi.org/10.1117/1.OE.59.7.073103.

    Article  Google Scholar 

  41. Wang W, Zhou Z, Liu H, Xie G. MSDRN: pansharpening of multispectral images via multi-scale deep residual network. Remote Sens. 2021;13(6):1200. https://doi.org/10.3390/rs13061200.

    Article  Google Scholar 

  42. Zhang J, Pan J, Ren J, Song Y, Yang MH. Dynamic scene deblurring using spatially variant recurrent neural networks. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition(CVPR); 2018. p. 2521–2529. https://doi.org/10.1109/CVPR.2018.00267.

  43. Tao X, Gao H, Shen X, Wang J, Jia J. Scale-recurrent network for deep image deblurring. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition(CVPR); 2018. p. 8174–8182. https://doi.org/10.1109/CVPR.2018.00853.

  44. Cai B, Xu X, Jia K, Qing C, Tao D. Dehazenet: an end-to-end system for single image haze removal. IEEE Trans Image Process. 2016;25(11):5187–98. https://doi.org/10.1109/TIP.2016.2598681.

    Article  MathSciNet  Google Scholar 

  45. Ren W, Liu S, Zhang H, Pan J, Cao X, Yang M-H. Single image dehazing via multi-scale convolutional neural network. In: 2016 European Conference on Computer Vision(ECCV); 2016. p. 154–169. https://doi.org/10.1007/978-3-319-46475-6_10.

  46. Zhang H, Patel VM. Densely connected pyramid dehazing network. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition(CVPR); 2018. p. 3194–3203. https://doi.org/10.1109/CVPR.2018.00337.

  47. Guo C, Li C, Gou J, Loy CC, Hou J, Kwong S, Cong R. Zero-reference deep curve estimation for low-light image enhancement. In: 2020 IEEE Conference on Computer Vision and Pattern Recognition(CVPR); 2020. p. 1780–1789. https://doi.org/10.1109/CVPR42600.2020.00185.

  48. Li M, Liu J, Yang W, Sun X, Guo Z. Structure-revealing low-light image enhancement via robust Retinex model. IEEE Trans Image Process. 2018;27(6):2828–41. https://doi.org/10.1109/TIP.2018.2810539.

    Article  MathSciNet  Google Scholar 

  49. Quan R, Yu X, Liang Y, Yang Y. Removing raindrops and rain streaks in one go. In: 2021 IEEE Conference on Computer Vision and Pattern Recognition(CVPR); 2021. p. 9147–9156. https://doi.org/10.1109/CVPR46437.2021.00903.

  50. Wu H, Qu Y, Lin S, Zhou J, Qiao R, Zhang Z, Xie Y, Ma L. Contrastive learning for compact single image dehazing. In: 2021 IEEE Conference on Computer Vision and Pattern Recognition(CVPR); 2021. p. 10551–10560. https://doi.org/10.1109/CVPR.2021.01041.

  51. Li X, Chen M, Nie F, Wang Q. A multiview-based parameter free framework for group detection. In: Proceedings of the AAAI Conference on Artificial Intelligence; 2017. p. 4147–4153. https://doi.org/10.1609/aaai.v31i1.11208.

  52. Qi Y, Zhang S, Jiang F, Zhou H, Tao D, Li X. Siamese local and global networks for robust face tracking. IEEE Trans Image Process. 2020;29:9152–64. https://doi.org/10.1109/TIP.2020.3023621.

    Article  Google Scholar 

  53. Li X, Chen M, Wang Q. Quantifying and detecting collective motion in crowd scenes. IEEE Trans Image Process. 2020;29:5571–83. https://doi.org/10.1109/TIP.2020.2985284.

    Article  Google Scholar 

  54. Pan J, Lin Z, Su Z, Yang MH. Robust kernel estimation with outliers handling for image deblurring. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). 2016. https://doi.org/10.1109/CVPR.2016.306.

  55. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process. 2004;13(4):600–12. https://doi.org/10.1109/TIP.2003.819861.

    Article  Google Scholar 

  56. Mittal A, Moorthy AK, Bovik AC. No-reference image quality assessment in the spatial domain. IEEE Trans Image Process. 2012;21(12):4695–780. https://doi.org/10.1109/TIP.2012.2214050.

    Article  MathSciNet  Google Scholar 

  57. Mittal A, Fellow IEEE, Soundararajan R, Bovik AC. Making a “completely blind" image quality analyzer. IEEE Signal Process Lett. 2013;20(3):209–12. https://doi.org/10.1109/LSP.2012.2227726.

Download references

Acknowledgements

Thank JEO Assistant Vincent salvo for taking the trouble to point out our format problems.

Funding

None.

Author information

Authors and Affiliations

  1. Key Laboratory of Spectral Imaging Technology CAS, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an, 710119, Shaanxi, China

    Yuetian Shi & Nan Wang

  2. University of Chinese Academy of Sciences, Beijing, 100049, Beijing, China

    Yuetian Shi & Nan Wang

  3. SenseTime Researchs, Shenzhen, 518000, Guangdong, China

    Bin Fu

  4. School of Computer and Artificial Intelligence, Wuhan University of Technology, Wuhan, 430000, Hubei, China

    Yaxiong Chen

  5. School of Telecommunication and Information Engineering, Xi’an University of Posts and Telecommunications, Xi’an, 710119, Shaanxi, China

    Jie Fang

  6. Corporation of Shaanxi Wukong Clouds Information and Technology, Xi’an, 710119, Shaanxi, China

    Jie Fang

Authors
  1. Yuetian Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bin Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yaxiong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jie Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jie Fang.

Ethics declarations

Ethics Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed Consent

None human participants.

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shi, Y., Fu, B., Wang, N. et al. Multispectral Image Quality Improvement Based on Global Iterative Fusion Constrained by Meteorological Factors. Cogn Comput 16, 404–424 (2024). https://doi.org/10.1007/s12559-023-10207-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-023-10207-7

Keywords

Search

Navigation