Abstract
The passive millimeter wave (PMMW) imaging sensor can generate images using the passive detection of the natural millimeter wave radiation from a scene. Despite the advantages of PMMW images in detecting concealed objects under clothing, they have lower resolution and fewer details than visible images. This paper proposes a new method to fuse PMMW and visible images to highlight concealed objects on the human body while preserving the details of the visible images. In this method, the PMMW image is initially segmented into three binary images, target, foreground, and background, utilizing an innovative segmentation algorithm that incorporates histogram-based thresholding and the generation of a saliency map image. Subsequently, the visible and PMMW images are individually decomposed into base and detail subbands using the new Gradient Transform (GT). Then, by individually fusing the base and detail subbands of the PMMW and visible images using innovative L2-norm weighting criteria, the fused image’s base and detail subbands are produced. Based on these criteria, between the two corresponding subbands of the input images, the subband with more detail contributes more to the final fused subband. Finally, the fused image is generated by applying the inverse GT to the newly generated fused subbands. Experimental results demonstrate a notable enhancement in terms of evaluation criteria like \({Q_{AB/F}}\) and MI, surpassing the most recent algorithms in this field.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11760-024-03761-6/MediaObjects/11760_2024_3761_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11760-024-03761-6/MediaObjects/11760_2024_3761_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11760-024-03761-6/MediaObjects/11760_2024_3761_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11760-024-03761-6/MediaObjects/11760_2024_3761_Fig4_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11760-024-03761-6/MediaObjects/11760_2024_3761_Fig5_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11760-024-03761-6/MediaObjects/11760_2024_3761_Fig6_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11760-024-03761-6/MediaObjects/11760_2024_3761_Fig7_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11760-024-03761-6/MediaObjects/11760_2024_3761_Fig8_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11760-024-03761-6/MediaObjects/11760_2024_3761_Fig9_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11760-024-03761-6/MediaObjects/11760_2024_3761_Fig10_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11760-024-03761-6/MediaObjects/11760_2024_3761_Fig11_HTML.png)
Similar content being viewed by others
Data Availability
No datasets were generated or analysed during the current study.
References
Pang, L., et al.: Real-time concealed object detection from passive millimeter wave images based on the yolov3 algorithm. Sensors 20(6), 1678 (2020)
Cheng, Y., et al.: Multi-polarization passive millimeter-wave imager and outdoor scene imaging analysis for remote sensing applications. optics express. optics express 26(16), 20145–20159 (2018)
Dillon, T.E., et al.: Passive, real-time millimeter wave imaging for degraded visual environment mitigation. InDegraded Visual Environments: Enhanced, Synthetic, and External Vision Solutions 2015(9471), 10–18 (2015)
Ramandi, V.Y.: Applying sgd optimization algorithm method for detecting and localizing of concealed objects in passive millimeter-wave images. Turkish Journal of Computer and Mathematics Education (TURCOMAT) 12(13), 4526–4533 (2021)
Su, J., et al.: Material clustering using passive millimeter-wave polarimetric imagery. IEEE Photonics Journal 11(1), 1–9 (2018)
Salmon, N.A.: Outdoor passive millimeter-wave imaging: Phenomenology and scene simulation. IEEE transactions on antennas and propagation 66(2), 897–908 (2017)
Guo, L., Qin, S.: High-performance detection of concealed forbidden objects on human body with deep neural networks based on passive millimeter wave and visible imagery. Journal of Infrared, Millimeter, and Terahertz Waves 40, 314–347 (2018)
Zhao, Y., et al.: A novel near field image reconstruction method based on beamforming technique for real-time passive millimeter wave imaging. IEEE Access 10, 32879–32888 (2022)
Meng, Y., et al.: Passive millimeter wave imaging system for public security check. In2017 International Applied Computational Electromagnetics Society Symposium, Suzhou, China (August 1–4, 2017). IEEE
Appleby, R., et al.: Millimeter wave imaging: a historical review. Passive and Active Millimeter-Wave Imaging XX 10189, 1018902 (2017)
Shi, Y., et al.: Hinrdnet: A half instance normalization residual dense network for passive millimetre wave image restoration. Infrared Physics & Technology 132, 104722 (2023)
Sun, D., et al.: Blind deblurring and denoising via a learning deep cnn denoiser prior and an adaptive l0-regularised gradient prior for passive millimetre-wave images. IET Image Processing 14(17), 4774–4784 (2020)
Xie, P., et al.: Resolution enhancement for millimeter-wave radar roi image with bayesian compressive sensing. Sensors 22(15), 5757 (2022)
Mansoori, M.A., et al.: Regularization-based semi-blind image deconvolution using an improved function for pmmw images application. Journal of Circuits, Systems and Computers 27(7), 18501071–185010725 (2018)
AminiRad, R., Afifi, A., Fahimifar, M.H.: Quality improving of millimeter wave images using fusion with visible images. Electronic and Cyber Defense 9(4), 77–86 (2022)
Zhou, Y., et al.: A survey of multi-focus image fusion methods. Applied Sciences 12(12), 6281 (2022)
Wang, H., et al.: Infrared and visible image fusion based on multi?channel convolutional neural network. IET image processing 16(6), 1575–1584 (2022)
Faragallah, O.S., et al.: A comprehensive survey analysis for present solutions of medical image fusion and future directions. IEEE Access 9, 11358–11371 (2020)
Zhang, L., et al.: Image fusion of pmmw and optical images for concealed object detection. Journal of Physics: Conference Seriesr. (2021). IOP Publishing
Bavirisetti, D.P., Dhuli, R.: Two-scale image fusion of visible and infrared images using saliency detection. Infrared Physics & Technology 76, 52–64 (2016)
Naidu, V.P., Raol, J.R.: Pixel-level image fusion using wavelets and principal component analysis. Defence science journal 58(3), 338 (2008)
Zhao, Z., et al.: Bayesian fusion for infrared and visible images. Signal Processing 177, 10734 (2020)
Wang, Z., et al.: Review of image fusion based on pulse-coupled neural network. Archives of Computational Methods in Engineering 23, 659–671 (2016)
Jial, Y., et al.: A multi-focus image fusion algorithm using modified adaptive PCNN model. 12th international conference on natural computation, fuzzy systems and knowledge discovery (ICNC-FSKD), Changsha, China ,13–15 August 2016 (2016)
Lee, H., et al.: Image fusion of visual and millimeter wave images for concealed object detection. 34th International Conference on Infrared, Millimeter, and Terahertz Waves, Busan, Korea (South), 21–25 September 2009 (2009)
Arun, P.S., et al.: Despeckling of OCT images using DT-CWT based fusion technique. Optik 263, 169332 (2022)
Xiong, J., et al.: A Novel Image Fusion Algorithm for Visible and PMMW Images based on Clustering and NSCT. InMATEC Web of Conferences, 2016 (2016)
Huang, Y., et al.: Fusion of visible and infrared image based on stationary tetrolet transform. In2017 32nd Youth Academic Annual Conference of Chinese Association of Automation (YAC), Hefei, China, 19–21 May 2017 (2017)
Xing, X., et al.: Infrared and visible image fusion based on nonlinear enhancement and nsst decomposition. EURASIP Journal on Wireless Communications and Networking 2020, 1–17 (2020)
Vijayarajan, R., Muttan, S.: Discrete wavelet transform based principal component averaging fusion for medical images. AEU-International Journal of Electronics and Communications 69(6), 896–902 (2015)
Ramlal, S.D., et al.: Multimodal medical image fusion using non-subsampled shearlet transform and pulse coupled neural network incorporated with morphological gradient. Signal, Image and Video Processing 12, 1479–1487 (2018)
Yin, M., et al.: Medical image fusion with parameter-adaptive pulse coupled neural network in nonsubsampled shearlet transform domain. IEEE Transactions on Instrumentation and Measurement 68(1), 49–64 (2018)
Li, Y., et al.: A visible and passive millimeter wave image fusion algorithm based on pulse-coupled neural network in tetrolet domain for early risk warning. Mathematical Problems in Engineering 2018 (2018)
Hadinejad, I., et al.: An optimum method for noise reduction and quality improvement of the passive millimeter wave images based on nonsubsampled shearlet transform and improved adaptive median filter. Journal of Information and Communication Technology in Policing 3(12), 30–43 (2023)
A., T.: TNO Image fusion dataset. https://figshare.com/articles/dataset/ TNO_Image_Fusion_Dataset/1008029 (2014)
Tan, W., et al.: Multi-modal brain image fusion based on multi-level edge-preserving filtering. Biomedical Signal Processing and Control 64, 102280 (2021)
Li, X., et al.: Laplacian redecomposition for multimodal medical image fusion. IEEE Transactions on Instrumentation and Measurement 69(9), 6880–6890 (2020)
He, K., et al.: Fidelity-driven optimization reconstruction and details preserving guided fusion for multi-modality medical image. IEEE Transactions on Multimedia 25, 4943–4957 (2022)
Polinati, S., Dhul, i.R.: Multimodal medical image fusion using empirical wavelet decomposition and local energy maxima. Optik 205, 163947 (2020)
Hermessi, H., et al.: Multimodal medical image fusion review: Theoretical background and recent advances. Signal Processing 183, 108036 (2021)
Haghighat, M., Razian, M.A.: Fast-FMI: Non-reference image fusion metric. In2014 IEEE 8th International Conference on Application of Information and Communication Technologies (AICT), Astana, Kazakhstan, 15–17 October 2014 (2014)
Kaur, H., et al.: Image fusion techniques: a survey. Archives of computational methods in Engineering 28, 4425–4447 (2021)
Li, G., et al.: An infrared and visible image fusion method based on multi-scale transformation and norm optimization. Information Fusion 71, 109–129 (2021)
Author information
Authors and Affiliations
Contributions
Iraj Hadinejad: Conceptualization, methodology, formal analysis, software and writing the main manuscript text. Mohammad Amin Amiri and Mohammad Hossein Fahimifar: Investigation, supervision, data curation, writing, reviewing and editing. All authors reviewed the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no Conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Hadinejad, I., Amiri, M.A. & Fahimifar, M.H. Passive millimeter wave and visible image fusion using concealed object detection and gradient transform. SIViP 19, 181 (2025). https://doi.org/10.1007/s11760-024-03761-6
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11760-024-03761-6