Skip to main content

Advertisement

Springer Nature Link
Log in

Infrared image super-resolution method based on dual-branch deep neural network

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Infrared image has lower resolution, lower contrast, and less detail than visible image, which causes its super-resolution (SR) more difficult than visible image. This paper presents an approach based on a deep neural network that comprises an image SR branch and a gradient SR branch to reconstruct high-quality SR image from single-frame infrared image. The image SR branch reconstructs the SR image from the initial low-resolution infrared image using a basic structure similar to the enhanced SR generative adversarial network (ESRGAN). The gradient SR branch removes haze, extracts the gradient map, and reconstructs the SR gradient map. To obtain more natural SR image, a fusion block based on attention mechanism is adopted between these branches. To preserve the geometric structure, gradient L1 loss and gradient GAN loss are defined and added. Experimental results on a public infrared image dataset demonstrate that, compared with the current SR methods, the proposed method is more natural and realistic, and can better preserve the structures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Fangzhe, N., Yurong, Q., Yanni, X., et al.: Survey of single image super resolution based on deep learning. Appl. Res. Comput. 37(02), 321–326 (2020)

    Google Scholar 

  2. Dong, C., Loy, C.C., He, K. et al.: Learning a deep convolutional network for image super-Resolution. Computer Vision-ECCV 2014. [S.I.]: Springer International Publishing, pp. 184–199 (2014)

  3. Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional network, pp. 1646–1654 (2015)

  4. Zhang, K., Zuo, W., Gu, S. et al.: Learning deep CNN denoiser prior for image restoration. In: Proc of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2808–2817 (2017)

  5. Tai, Y., Yang, J., Liu, X. et al.: MemNet:a persistent memory network for image restoration. In: Proc of IEEE International Conference on Computer Vision.[S.I.]: IEEEComputer Society, pp. 4549–4557 (2017)

  6. Lai, W., Huang, J., Ahuja, N. et al.: Deep laplacian pyramid networks for fast and accurate super-resolution. In: Proc of IEEE International Conference on Computer Vision. [S.I.]: IEEE Computer Society, pp. 5835–5843 (2017)

  7. Tai, Y., Yang, J., Liu, X.: Image super-resolution via deep recursive residual network. In: Proc of IEEE Conference on Computer Vision and Pattern Recognition. [S.I.]: IEEE Computer Society, pp. 2790–2798 (2017)

  8. Sumei, Li., Fan, Ru., Guoqing, L., et al.: A two-channel convolutional neural network for image super-resolution. Neurocomputing 275, 267–277 (2018)

    Article  Google Scholar 

  9. Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: Proc of European Conference on Computer Vision. Berlin: Springer, pp. 392–407 (2016)

  10. Xiaojiao, M., Chunhua, S., Yubin, Y.: Image restoration using convolutional auto-encoders with symmetric skip connections. Res. Gate 6, 391 (2016)

    Google Scholar 

  11. Shi, W., Caballero, J., Huszár, F. et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1874–1883 (2016)

  12. Ledig, C., Theis, L., Huszar, F. et al.: Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. 2016(9):105–114.

  13. Lim, B., Son, S., Kim, H. et al.: Enhanced deep residual networks for single image super-resolution. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE Computer Society, pp. 1132–1140 (2017)

  14. Li, L., Tang, J., Ye, Z., et al.: Unsupervised face super-resolution via gradient enhancement and semantic guidance. Vis. Comput. 37, 2855–2867 (2021)

    Article  Google Scholar 

  15. Tong, T., Li, G., Liu, X. et al.: Image super-resolution using dense skip connections. In: 2017 IEEE International Conference on Computer Vision (ICCV). IEEE Computer Society, pp. 4809–4817 (2017)

  16. Amaranageswarao, G., Deivalakshmi, S., Ko, S.B.: Joint restoration convolutional neural network for low-quality image super resolution. Vis. Comput. 38(1), 31–50 (2022)

    Article  Google Scholar 

  17. Danya, Z., Yepeng, L., Xuemei, Li., et al.: Single-image super-resolution based on local biquadratic spline with edge constraints and adaptive optimization in transform domain. Vis. Comput. 38(1), 119–134 (2022)

    Article  Google Scholar 

  18. Zhang, Y., Tian, Y., Kong, Y. et al.: Residual dense network for image restoration. 2018(2):180.

  19. Zhou, F., Li, X., Li, Z.: High-frequency details enhancing dense net for super-resolution. Neurocomputing 290, 34–42 (2018)

    Article  Google Scholar 

  20. Ledig, C., Theis, L., Huszar, F. et al.: Photo-realistic single image super-resolution using a generative adversarial network. Preprint https://arxiv.org/abs/1609.04802 (2016)

  21. Wang, X., Yu, K., Wu, S. et al.: ESRGAN: enhanced super-resolution generative adversarial networks. In: The European Conference on Computer Vision Workshops (ECCVW), pp. 1–23 (2018)

  22. Mao, R.: Single Infrared Image Super-resolution and Enhancement Based on Fusion ESRGAN and Gradient Network. 2020, Xian University of Technology.

  23. Wang, X., Zhang, K., Yan, J., et al.: Infrared image complexity metric for automatic target recognition based on neural network and traditional approach fusion. Arab. J. Sci. Eng. 45(4), 3245–3255 (2020)

    Article  Google Scholar 

  24. Ma, C., Rao, Y., Cheng, Y. et al.: Structure-Preserving Super Resolution with Gradient Guidance.2020, https://arxiv.org/abs/2003.13063

  25. Nayak, R., Balabantaray, B.K., Patra, D.: A new single-image super-resolution using efficient feature fusion and patch similarity in non-euclidean space. Arab. J. Sci. Eng. 45(12), 10261–10285 (2020)

    Article  Google Scholar 

  26. Kaiming, H., Jian, S., Xiaoou, T.: Single Image Haze Removal Using Dark Channel Prior. CVPR (2009)

  27. Gautam, A., Singh, S.: Neural style transfer combined with EfficientDet for thermal surveillance. The Visual Computer, pp. 1–17 (2021)

  28. Songchen, H., Changxin, H., Wei, Li., et al.: An improved dehazing algorithm based on near infrared image. Adv. Eng. Sci. 50(2), 347–356 (2018)

    Google Scholar 

  29. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P., et al.: Image quality assessment: from error visibility to structural similarity. TIP 13(4), 600–612 (2004)

    ADS  Google Scholar 

  30. Yochai, B., Roey, M., Radu, T., Tomer, M., Lihi, Z-M.: The 2018 pirm challenge on perceptual image super-resolution. In: ECCV, Springer, pp. 334–355 (2018)

  31. Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston (2015)

Download references

Acknowledgements

This work was supported by Key Laboratory Fund of Basic Strengthening Program (JKWATR-210503), Changsha Municipal Natural Science Foundation (kq2202067), and the Basic Science and Technology Research Project of the National Key Laboratory of Science and Technology on Automatic Target Recognition of Scientific Research under Grant (WDZC20205500209).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Changsha University, Changsha, 410003, China

    Huang Zhijian

  2. ATR Key Laboratory, School of Electronic Science, National University of Defense Technology, Changsha, China

    Hui Bingwei

  3. Hunan Province Key Laboratory of Industrial Internet Technology and Security, Changsha, China

    Sun Shujin

  4. College of Meteorology and Oceanology, National University of Defense Technology, Changsha, China

    Ma Yanxin

Authors
  1. Huang Zhijian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hui Bingwei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sun Shujin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ma Yanxin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huang Zhijian.

Ethics declarations

Conflict of interest

We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhijian, H., Bingwei, H., Shujin, S. et al. Infrared image super-resolution method based on dual-branch deep neural network. Vis Comput 40, 1673–1684 (2024). https://doi.org/10.1007/s00371-023-02878-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-023-02878-y

Keywords