Skip to main content
SpringerLink
Log in

Infrared image super-resolution method for edge computing based on adaptive nonlocal means

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Infrared images generally suffer from low resolutions, which restricts the further data mining of these images. Super-resolution (SR) technology effectively improves the spatial resolution of the infrared image without changing the existing hardware imaging equipment. Therefore, it is a promising computational imaging approach with resource-limited edge devices. In this paper, an adaptive-threshold nonlocal means (NLM)-based SR algorithm is proposed. Specifically, an image quality assessment index of infrared SR results is designed and introduced into the NLM reconstruction algorithm. On the one hand, it is used as a threshold to determine the iterative convergence condition of the algorithm; on the other hand, it is used as an evaluation standard to select the best reconstructed HR image among multiple output results. A GPU acceleration strategy is also proposed to ensure the high efficiency of the edge computing process for reducing the computational time. Experimental results demonstrate that the algorithm realizes the adaptive iteration of the NLM-based SR of the infrared images, and it significantly improves the geometric structures and detail recognition of the original input LR images. The sawtooth and ringing effects of the algorithm are less, and its objective evaluation indexes are also significantly improved compared with those of other SR algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Filippini C, Perpetuini D, Cardone D et al (2020) Thermal infrared imaging-based affective computing and its application to facilitate human robot interaction: a review. Appl Sci 10(8):1–23

    Article  Google Scholar 

  2. Dai X, Yuan X, Wei X (2021) TIRNet: object detection in thermal infrared images for autonomous driving. Appl Intell 51(3):1244–1261

    Article  Google Scholar 

  3. Xuan RP, Xiong Y, Brietzke A et al (2020) Thermal infrared imaging based facial temperature in comparison to ear temperature during a real-driving scenario. J Therm Biol 96(5):1–8

    Google Scholar 

  4. Pavlovetc IM, Aleshire K, Hartland GV et al (2020) Approaches to mid-infrared, super-resolution imaging and spectroscopy. Phys Chem Chem Phys 22(8):4313–4325

    Article  Google Scholar 

  5. Xiong Z, Yu Q, Sun T et al (2020) Super-resolution reconstruction of real infrared images acquired with unmanned aerial vehicle. PLoS One 15(6):1–18

    Article  Google Scholar 

  6. Anwar S, Khan S, Barnes N (2020) A deep journey into super-resolution: a survey. ACM Comput Surv (CSUR) 53(3):1–34

    Article  Google Scholar 

  7. Singh A, Singh J (2020) Survey on single image based super-resolution—implementation challenges and solutions. Multimed Tools Appl 79(3):1641–1672

    Article  Google Scholar 

  8. Liu X, Chen L, Wang W et al (2018) Robust multi-frame super-resolution based on spatially weighted half-quadratic estimation and adaptive BTV regularization. IEEE Trans Image Process 27(10):4971–4986

    Article  MathSciNet  Google Scholar 

  9. Mandanici E, Tavasci L, Corsini F et al (2019) A multi-image super-resolution algorithm applied to thermal imagery. Appl Geomat 11(3):215–228

    Article  Google Scholar 

  10. Salvetti F, Mazzia V, Khaliq A et al (2020) Multi-image super resolution of remotely sensed images using residual attention deep neural networks. Remote Sens 12(14):2207

    Article  Google Scholar 

  11. Nguyen N L, Anger J, Davy A, et al (2021) SELF-SUPERVISED MULTI-IMAGE SUPER-RESOLUTION FOR PUSH-FRAME SATELLITE IMAGES. In: Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 1121–1131

  12. Farsiu S, Robinson D, Elad M, et al (2003) Robust shift and add approach to super-resolution. In: Proceedings of SPIE the International Society for Optical Engineering 5, Vol. 5203. SPIE, pp 121–130

  13. Fernández Á, Rabin N, Fishelov D et al (2020) Auto-adaptive multi-scale Laplacian Pyramids for modeling non-uniform data. Eng Appl Artif Intell 93:103682

    Article  Google Scholar 

  14. Irani M, Peleg S (1993) Motion analysis for image enhancement: Resolution, occlusion, and transparency. J Vis Commun Image Represent 4(4):324–335

    Article  Google Scholar 

  15. Tom BC, Katsaggelos AK (2001) Resolution enhancement of monochrome and color video using motion compensation. IEEE Trans Image Process 10(2):278–287

    Article  Google Scholar 

  16. Lu Y, Imamura M (2002) Pyramid-based super-resolution of the undersampled and sub-pixel shifted image sequence. Int J Imag Syst Technol 12(6):254–263

    Article  Google Scholar 

  17. Nayak R, Patra D (2017) An edge preserving IBP based super resolution image reconstruction using P-spline and MuCSO-QPSO algorithm. Microsyst Technol 23(3):553–569

    Article  Google Scholar 

  18. Zhang M, Desrosiers C, Qiang Q, et al (2016) Medical image super-resolution with non-local embedding sparse representation and improved IBP. In: Proceedings of 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp 888–892

    Google Scholar 

  19. Stark H, Oskoui P (1989) High-resolution image recovery from image-plane arrays using convex projections. JOSA A 6(11):1715–1726

    Article  Google Scholar 

  20. Tang Z, Deng M, Xiao C, et al (2011) Projection onto convex sets super-resolution image reconstruction based on wavelet bi-cubic interpolation. In: Proceedings of 2011 International Conference on Electronic & Mechanical Engineering and Information Technology. IEEE, pp 351–354

    Chapter  Google Scholar 

  21. Shen W, Fang L, Chen X et al (2014) Projection onto convex sets method in space-frequency domain for super resolution. JCP 9(8):1959–1966

    Google Scholar 

  22. Fan C, Wu C, Li G et al (2017) Projections onto convex sets super-resolution reconstruction based on point spread function estimation of low-resolution remote sensing images. Sensors 17(2):362

    Article  Google Scholar 

  23. Fuhry M, Reichel L (2012) A new Tikhonov regularization method. Numer Algorithms 59(3):433–445

    Article  MathSciNet  Google Scholar 

  24. Chan T, Esedoglu S, Park F et al (2005) Recent developments in total variation image restoration. Math Models Comput Vis 17(2):17–31

    Google Scholar 

  25. Li X, Huang J, Deng LJ et al (2019) Bilateral filter based total variation regularization for sparse hyper spectral image unmixing. Inf Sci 504:334–353

    Article  Google Scholar 

  26. Shen H, Zhang L, Huang B et al (2007) A map approach for joint motion estimation, segmentation, and super resolution. IEEE Trans Image process 16(2):479–490

    Article  MathSciNet  Google Scholar 

  27. Lu X, Yuan Y, Yan P (2013) Image super-resolution via double sparsity regularized manifold learning. IEEE Trans Circuits Syst Video Technol 23(12):2022–2033

    Article  Google Scholar 

  28. Fournier C, Jolivet F, Denis L et al (2017) Pixel super-resolution in digital holography by regularized reconstruction. Appl Opt 56(1):69–77

    Article  Google Scholar 

  29. Buades A, Coll B, Morel J M (2005) A NON-LOCAL ALGORITHM FOR IMAGE DENOISING. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, USA: IEEE, 60–65.

  30. Protter M, Elad M, Takeda H et al (2008) Generalizing the nonlocal-means to super-resolution reconstruction. IEEE Trans Image Process 18(1):36–51

    Article  MathSciNet  Google Scholar 

  31. Yu H, Chen F, Zhang Z et al (2013) Single infrared image super-resolution combining non-local means with kernel regression. Infrared Phys Technol 61:50–59

    Article  Google Scholar 

  32. Li Y, Li X, Fu Z (2018) Modified non-local means for super-resolution of hybrid videos. Comput Vis Image Underst 168:64–78

    Article  Google Scholar 

  33. Mandal S, Bhavsar A, Sao AK (2017) Noise adaptive super-resolution from single image via non-local mean and sparse representation. Signal Process 132:134–149

    Article  Google Scholar 

  34. Salmon J (2009) On two parameters for denoising with non-local means. IEEE Signal Process Lett 17(3):269–272

    Article  Google Scholar 

  35. Moraes T, Amorim P, Da Silva JV et al (2020) Medical image interpolation based on 3D Lanczos filtering. Comput Methods Biomech Biomed Eng Imag Vis 8(3):294–300

    Article  Google Scholar 

  36. Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain. IEEE Trans Image Process 21(12):4695–4708

    Article  MathSciNet  Google Scholar 

  37. Jian L, Wang C, Liu Y et al (2013) Parallel data mining techniques on graphics processing unit with compute unified device architecture (CUDA). J Supercomput 64(3):942–967

    Article  Google Scholar 

  38. He Z, Cao Y, Dong Y et al (2018) Single-image-based non-uniformity correction of uncooled long-wave infrared detectors: a deep-learning approach. Appl Opt 57(18):155–164

    Article  Google Scholar 

Download references

Funding

This work was supported in part by the Science and Technology on Near-Surface Detection Laboratory Foundation of China (No. Grant 614241409041317), and in part by Major Project for Special Technology Innovation of Hubei Province (No. Grant 2018AAA029).

Author information

Authors and Affiliations

  1. Electronic Information School, Wuhan University, Wuhan, 430072, China

    Tao Sun, Zhengqiang Xiong & Zixian Wei

  2. State Key Laboratory of Bridge Structure Health and Safety, Wuhan, 430034, China

    Zhengxing Wang

Authors
  1. Tao Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhengqiang Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zixian Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhengxing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhengqiang Xiong.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, T., Xiong, Z., Wei, Z. et al. Infrared image super-resolution method for edge computing based on adaptive nonlocal means. J Supercomput 78, 6717–6738 (2022). https://doi.org/10.1007/s11227-021-04141-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-021-04141-4

Keywords

Search

Navigation