Abstract
Infrared images generally suffer from low resolutions, which restricts the further data mining of these images. Super-resolution (SR) technology effectively improves the spatial resolution of the infrared image without changing the existing hardware imaging equipment. Therefore, it is a promising computational imaging approach with resource-limited edge devices. In this paper, an adaptive-threshold nonlocal means (NLM)-based SR algorithm is proposed. Specifically, an image quality assessment index of infrared SR results is designed and introduced into the NLM reconstruction algorithm. On the one hand, it is used as a threshold to determine the iterative convergence condition of the algorithm; on the other hand, it is used as an evaluation standard to select the best reconstructed HR image among multiple output results. A GPU acceleration strategy is also proposed to ensure the high efficiency of the edge computing process for reducing the computational time. Experimental results demonstrate that the algorithm realizes the adaptive iteration of the NLM-based SR of the infrared images, and it significantly improves the geometric structures and detail recognition of the original input LR images. The sawtooth and ringing effects of the algorithm are less, and its objective evaluation indexes are also significantly improved compared with those of other SR algorithms.
Similar content being viewed by others
References
Filippini C, Perpetuini D, Cardone D et al (2020) Thermal infrared imaging-based affective computing and its application to facilitate human robot interaction: a review. Appl Sci 10(8):1–23
Dai X, Yuan X, Wei X (2021) TIRNet: object detection in thermal infrared images for autonomous driving. Appl Intell 51(3):1244–1261
Xuan RP, Xiong Y, Brietzke A et al (2020) Thermal infrared imaging based facial temperature in comparison to ear temperature during a real-driving scenario. J Therm Biol 96(5):1–8
Pavlovetc IM, Aleshire K, Hartland GV et al (2020) Approaches to mid-infrared, super-resolution imaging and spectroscopy. Phys Chem Chem Phys 22(8):4313–4325
Xiong Z, Yu Q, Sun T et al (2020) Super-resolution reconstruction of real infrared images acquired with unmanned aerial vehicle. PLoS One 15(6):1–18
Anwar S, Khan S, Barnes N (2020) A deep journey into super-resolution: a survey. ACM Comput Surv (CSUR) 53(3):1–34
Singh A, Singh J (2020) Survey on single image based super-resolution—implementation challenges and solutions. Multimed Tools Appl 79(3):1641–1672
Liu X, Chen L, Wang W et al (2018) Robust multi-frame super-resolution based on spatially weighted half-quadratic estimation and adaptive BTV regularization. IEEE Trans Image Process 27(10):4971–4986
Mandanici E, Tavasci L, Corsini F et al (2019) A multi-image super-resolution algorithm applied to thermal imagery. Appl Geomat 11(3):215–228
Salvetti F, Mazzia V, Khaliq A et al (2020) Multi-image super resolution of remotely sensed images using residual attention deep neural networks. Remote Sens 12(14):2207
Nguyen N L, Anger J, Davy A, et al (2021) SELF-SUPERVISED MULTI-IMAGE SUPER-RESOLUTION FOR PUSH-FRAME SATELLITE IMAGES. In: Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 1121–1131
Farsiu S, Robinson D, Elad M, et al (2003) Robust shift and add approach to super-resolution. In: Proceedings of SPIE the International Society for Optical Engineering 5, Vol. 5203. SPIE, pp 121–130
Fernández Á, Rabin N, Fishelov D et al (2020) Auto-adaptive multi-scale Laplacian Pyramids for modeling non-uniform data. Eng Appl Artif Intell 93:103682
Irani M, Peleg S (1993) Motion analysis for image enhancement: Resolution, occlusion, and transparency. J Vis Commun Image Represent 4(4):324–335
Tom BC, Katsaggelos AK (2001) Resolution enhancement of monochrome and color video using motion compensation. IEEE Trans Image Process 10(2):278–287
Lu Y, Imamura M (2002) Pyramid-based super-resolution of the undersampled and sub-pixel shifted image sequence. Int J Imag Syst Technol 12(6):254–263
Nayak R, Patra D (2017) An edge preserving IBP based super resolution image reconstruction using P-spline and MuCSO-QPSO algorithm. Microsyst Technol 23(3):553–569
Zhang M, Desrosiers C, Qiang Q, et al (2016) Medical image super-resolution with non-local embedding sparse representation and improved IBP. In: Proceedings of 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp 888–892
Stark H, Oskoui P (1989) High-resolution image recovery from image-plane arrays using convex projections. JOSA A 6(11):1715–1726
Tang Z, Deng M, Xiao C, et al (2011) Projection onto convex sets super-resolution image reconstruction based on wavelet bi-cubic interpolation. In: Proceedings of 2011 International Conference on Electronic & Mechanical Engineering and Information Technology. IEEE, pp 351–354
Shen W, Fang L, Chen X et al (2014) Projection onto convex sets method in space-frequency domain for super resolution. JCP 9(8):1959–1966
Fan C, Wu C, Li G et al (2017) Projections onto convex sets super-resolution reconstruction based on point spread function estimation of low-resolution remote sensing images. Sensors 17(2):362
Fuhry M, Reichel L (2012) A new Tikhonov regularization method. Numer Algorithms 59(3):433–445
Chan T, Esedoglu S, Park F et al (2005) Recent developments in total variation image restoration. Math Models Comput Vis 17(2):17–31
Li X, Huang J, Deng LJ et al (2019) Bilateral filter based total variation regularization for sparse hyper spectral image unmixing. Inf Sci 504:334–353
Shen H, Zhang L, Huang B et al (2007) A map approach for joint motion estimation, segmentation, and super resolution. IEEE Trans Image process 16(2):479–490
Lu X, Yuan Y, Yan P (2013) Image super-resolution via double sparsity regularized manifold learning. IEEE Trans Circuits Syst Video Technol 23(12):2022–2033
Fournier C, Jolivet F, Denis L et al (2017) Pixel super-resolution in digital holography by regularized reconstruction. Appl Opt 56(1):69–77
Buades A, Coll B, Morel J M (2005) A NON-LOCAL ALGORITHM FOR IMAGE DENOISING. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, USA: IEEE, 60–65.
Protter M, Elad M, Takeda H et al (2008) Generalizing the nonlocal-means to super-resolution reconstruction. IEEE Trans Image Process 18(1):36–51
Yu H, Chen F, Zhang Z et al (2013) Single infrared image super-resolution combining non-local means with kernel regression. Infrared Phys Technol 61:50–59
Li Y, Li X, Fu Z (2018) Modified non-local means for super-resolution of hybrid videos. Comput Vis Image Underst 168:64–78
Mandal S, Bhavsar A, Sao AK (2017) Noise adaptive super-resolution from single image via non-local mean and sparse representation. Signal Process 132:134–149
Salmon J (2009) On two parameters for denoising with non-local means. IEEE Signal Process Lett 17(3):269–272
Moraes T, Amorim P, Da Silva JV et al (2020) Medical image interpolation based on 3D Lanczos filtering. Comput Methods Biomech Biomed Eng Imag Vis 8(3):294–300
Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain. IEEE Trans Image Process 21(12):4695–4708
Jian L, Wang C, Liu Y et al (2013) Parallel data mining techniques on graphics processing unit with compute unified device architecture (CUDA). J Supercomput 64(3):942–967
He Z, Cao Y, Dong Y et al (2018) Single-image-based non-uniformity correction of uncooled long-wave infrared detectors: a deep-learning approach. Appl Opt 57(18):155–164
Funding
This work was supported in part by the Science and Technology on Near-Surface Detection Laboratory Foundation of China (No. Grant 614241409041317), and in part by Major Project for Special Technology Innovation of Hubei Province (No. Grant 2018AAA029).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Sun, T., Xiong, Z., Wei, Z. et al. Infrared image super-resolution method for edge computing based on adaptive nonlocal means. J Supercomput 78, 6717–6738 (2022). https://doi.org/10.1007/s11227-021-04141-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11227-021-04141-4