Skip to main content
Log in

Passive millimeter wave and visible image fusion using concealed object detection and gradient transform

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

The passive millimeter wave (PMMW) imaging sensor can generate images using the passive detection of the natural millimeter wave radiation from a scene. Despite the advantages of PMMW images in detecting concealed objects under clothing, they have lower resolution and fewer details than visible images. This paper proposes a new method to fuse PMMW and visible images to highlight concealed objects on the human body while preserving the details of the visible images. In this method, the PMMW image is initially segmented into three binary images, target, foreground, and background, utilizing an innovative segmentation algorithm that incorporates histogram-based thresholding and the generation of a saliency map image. Subsequently, the visible and PMMW images are individually decomposed into base and detail subbands using the new Gradient Transform (GT). Then, by individually fusing the base and detail subbands of the PMMW and visible images using innovative L2-norm weighting criteria, the fused image’s base and detail subbands are produced. Based on these criteria, between the two corresponding subbands of the input images, the subband with more detail contributes more to the final fused subband. Finally, the fused image is generated by applying the inverse GT to the newly generated fused subbands. Experimental results demonstrate a notable enhancement in terms of evaluation criteria like \({Q_{AB/F}}\) and MI, surpassing the most recent algorithms in this field.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data Availability

No datasets were generated or analysed during the current study.

References

  1. Pang, L., et al.: Real-time concealed object detection from passive millimeter wave images based on the yolov3 algorithm. Sensors 20(6), 1678 (2020)

    Article  MATH  Google Scholar 

  2. Cheng, Y., et al.: Multi-polarization passive millimeter-wave imager and outdoor scene imaging analysis for remote sensing applications. optics express. optics express 26(16), 20145–20159 (2018)

    Article  Google Scholar 

  3. Dillon, T.E., et al.: Passive, real-time millimeter wave imaging for degraded visual environment mitigation. InDegraded Visual Environments: Enhanced, Synthetic, and External Vision Solutions 2015(9471), 10–18 (2015)

    MATH  Google Scholar 

  4. Ramandi, V.Y.: Applying sgd optimization algorithm method for detecting and localizing of concealed objects in passive millimeter-wave images. Turkish Journal of Computer and Mathematics Education (TURCOMAT) 12(13), 4526–4533 (2021)

    MATH  Google Scholar 

  5. Su, J., et al.: Material clustering using passive millimeter-wave polarimetric imagery. IEEE Photonics Journal 11(1), 1–9 (2018)

    Article  MATH  Google Scholar 

  6. Salmon, N.A.: Outdoor passive millimeter-wave imaging: Phenomenology and scene simulation. IEEE transactions on antennas and propagation 66(2), 897–908 (2017)

    Article  MATH  Google Scholar 

  7. Guo, L., Qin, S.: High-performance detection of concealed forbidden objects on human body with deep neural networks based on passive millimeter wave and visible imagery. Journal of Infrared, Millimeter, and Terahertz Waves 40, 314–347 (2018)

    Article  MATH  Google Scholar 

  8. Zhao, Y., et al.: A novel near field image reconstruction method based on beamforming technique for real-time passive millimeter wave imaging. IEEE Access 10, 32879–32888 (2022)

    Article  Google Scholar 

  9. Meng, Y., et al.: Passive millimeter wave imaging system for public security check. In2017 International Applied Computational Electromagnetics Society Symposium, Suzhou, China (August 1–4, 2017). IEEE

  10. Appleby, R., et al.: Millimeter wave imaging: a historical review. Passive and Active Millimeter-Wave Imaging XX 10189, 1018902 (2017)

    Article  MATH  Google Scholar 

  11. Shi, Y., et al.: Hinrdnet: A half instance normalization residual dense network for passive millimetre wave image restoration. Infrared Physics & Technology 132, 104722 (2023)

    Article  MATH  Google Scholar 

  12. Sun, D., et al.: Blind deblurring and denoising via a learning deep cnn denoiser prior and an adaptive l0-regularised gradient prior for passive millimetre-wave images. IET Image Processing 14(17), 4774–4784 (2020)

    Article  MATH  Google Scholar 

  13. Xie, P., et al.: Resolution enhancement for millimeter-wave radar roi image with bayesian compressive sensing. Sensors 22(15), 5757 (2022)

    Article  MATH  Google Scholar 

  14. Mansoori, M.A., et al.: Regularization-based semi-blind image deconvolution using an improved function for pmmw images application. Journal of Circuits, Systems and Computers 27(7), 18501071–185010725 (2018)

    Article  MATH  Google Scholar 

  15. AminiRad, R., Afifi, A., Fahimifar, M.H.: Quality improving of millimeter wave images using fusion with visible images. Electronic and Cyber Defense 9(4), 77–86 (2022)

    Google Scholar 

  16. Zhou, Y., et al.: A survey of multi-focus image fusion methods. Applied Sciences 12(12), 6281 (2022)

    Article  MATH  Google Scholar 

  17. Wang, H., et al.: Infrared and visible image fusion based on multi?channel convolutional neural network. IET image processing 16(6), 1575–1584 (2022)

    Article  MATH  Google Scholar 

  18. Faragallah, O.S., et al.: A comprehensive survey analysis for present solutions of medical image fusion and future directions. IEEE Access 9, 11358–11371 (2020)

    Article  MATH  Google Scholar 

  19. Zhang, L., et al.: Image fusion of pmmw and optical images for concealed object detection. Journal of Physics: Conference Seriesr. (2021). IOP Publishing

  20. Bavirisetti, D.P., Dhuli, R.: Two-scale image fusion of visible and infrared images using saliency detection. Infrared Physics & Technology 76, 52–64 (2016)

    Article  MATH  Google Scholar 

  21. Naidu, V.P., Raol, J.R.: Pixel-level image fusion using wavelets and principal component analysis. Defence science journal 58(3), 338 (2008)

    Article  MATH  Google Scholar 

  22. Zhao, Z., et al.: Bayesian fusion for infrared and visible images. Signal Processing 177, 10734 (2020)

    Article  MATH  Google Scholar 

  23. Wang, Z., et al.: Review of image fusion based on pulse-coupled neural network. Archives of Computational Methods in Engineering 23, 659–671 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  24. Jial, Y., et al.: A multi-focus image fusion algorithm using modified adaptive PCNN model. 12th international conference on natural computation, fuzzy systems and knowledge discovery (ICNC-FSKD), Changsha, China ,13–15 August 2016 (2016)

  25. Lee, H., et al.: Image fusion of visual and millimeter wave images for concealed object detection. 34th International Conference on Infrared, Millimeter, and Terahertz Waves, Busan, Korea (South), 21–25 September 2009 (2009)

  26. Arun, P.S., et al.: Despeckling of OCT images using DT-CWT based fusion technique. Optik 263, 169332 (2022)

    Article  MATH  Google Scholar 

  27. Xiong, J., et al.: A Novel Image Fusion Algorithm for Visible and PMMW Images based on Clustering and NSCT. InMATEC Web of Conferences, 2016 (2016)

  28. Huang, Y., et al.: Fusion of visible and infrared image based on stationary tetrolet transform. In2017 32nd Youth Academic Annual Conference of Chinese Association of Automation (YAC), Hefei, China, 19–21 May 2017 (2017)

  29. Xing, X., et al.: Infrared and visible image fusion based on nonlinear enhancement and nsst decomposition. EURASIP Journal on Wireless Communications and Networking 2020, 1–17 (2020)

    Article  MATH  Google Scholar 

  30. Vijayarajan, R., Muttan, S.: Discrete wavelet transform based principal component averaging fusion for medical images. AEU-International Journal of Electronics and Communications 69(6), 896–902 (2015)

    MATH  Google Scholar 

  31. Ramlal, S.D., et al.: Multimodal medical image fusion using non-subsampled shearlet transform and pulse coupled neural network incorporated with morphological gradient. Signal, Image and Video Processing 12, 1479–1487 (2018)

    Article  MATH  Google Scholar 

  32. Yin, M., et al.: Medical image fusion with parameter-adaptive pulse coupled neural network in nonsubsampled shearlet transform domain. IEEE Transactions on Instrumentation and Measurement 68(1), 49–64 (2018)

    Article  MATH  Google Scholar 

  33. Li, Y., et al.: A visible and passive millimeter wave image fusion algorithm based on pulse-coupled neural network in tetrolet domain for early risk warning. Mathematical Problems in Engineering 2018 (2018)

  34. Hadinejad, I., et al.: An optimum method for noise reduction and quality improvement of the passive millimeter wave images based on nonsubsampled shearlet transform and improved adaptive median filter. Journal of Information and Communication Technology in Policing 3(12), 30–43 (2023)

    MATH  Google Scholar 

  35. A., T.: TNO Image fusion dataset. https://figshare.com/articles/dataset/ TNO_Image_Fusion_Dataset/1008029 (2014)

  36. Tan, W., et al.: Multi-modal brain image fusion based on multi-level edge-preserving filtering. Biomedical Signal Processing and Control 64, 102280 (2021)

    Article  MATH  Google Scholar 

  37. Li, X., et al.: Laplacian redecomposition for multimodal medical image fusion. IEEE Transactions on Instrumentation and Measurement 69(9), 6880–6890 (2020)

    Article  MATH  Google Scholar 

  38. He, K., et al.: Fidelity-driven optimization reconstruction and details preserving guided fusion for multi-modality medical image. IEEE Transactions on Multimedia 25, 4943–4957 (2022)

    Article  MATH  Google Scholar 

  39. Polinati, S., Dhul, i.R.: Multimodal medical image fusion using empirical wavelet decomposition and local energy maxima. Optik 205, 163947 (2020)

  40. Hermessi, H., et al.: Multimodal medical image fusion review: Theoretical background and recent advances. Signal Processing 183, 108036 (2021)

  41. Haghighat, M., Razian, M.A.: Fast-FMI: Non-reference image fusion metric. In2014 IEEE 8th International Conference on Application of Information and Communication Technologies (AICT), Astana, Kazakhstan, 15–17 October 2014 (2014)

  42. Kaur, H., et al.: Image fusion techniques: a survey. Archives of computational methods in Engineering 28, 4425–4447 (2021)

    Article  MATH  Google Scholar 

  43. Li, G., et al.: An infrared and visible image fusion method based on multi-scale transformation and norm optimization. Information Fusion 71, 109–129 (2021)

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Contributions

Iraj Hadinejad: Conceptualization, methodology, formal analysis, software and writing the main manuscript text. Mohammad Amin Amiri and Mohammad Hossein Fahimifar: Investigation, supervision, data curation, writing, reviewing and editing. All authors reviewed the manuscript.

Corresponding author

Correspondence to Mohammad Amin Amiri.

Ethics declarations

Conflict of interest

The authors declare no Conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hadinejad, I., Amiri, M.A. & Fahimifar, M.H. Passive millimeter wave and visible image fusion using concealed object detection and gradient transform. SIViP 19, 181 (2025). https://doi.org/10.1007/s11760-024-03761-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11760-024-03761-6

Keywords

Navigation