Abstract
Visual images have visual features that are convenient for human observation, while infrared images have thermal features. Image fusion of visual image and the infrared image is the integration of most conductive intrinsic features by still preserving its original visual information concealed in. Image fusion has significant importance in different areas application in special environment (e.g. military, security and in surveillance) and human vision. In recent years, due to the-state-of-the-art discriminative performance, deep learning has been greatly applied in field of image fusion due to its excellent feature extraction ability. This paper employs an innovative approach for image fusion for infrared and visual images using a pre-trained dense block neural network. The model uses an isolated encoder-decoder bi-layered deep learning models, each for visual images and infrared images. An intermediary novel fusion protocols was designed using a hybrid of fusion and L1_norm strategy to infuse both visual and infrared image features. Finally, the resulting images of decoders are made subject to a weighting average to obtained fusion resulting image. Based on subjective evaluation, the proposed model, outperform all existing state of the art literature studies, via different objective evaluation indicators.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles and news from researchers in related subjects, suggested using machine learning.References
Babenko A, Slesarev A, Chigorin A, Lempitsky V (2014). Neural codes for image retrieval. In: European conference on computer vision (pp. 584–599). Springer, Cham
Bavirisetti DP, Xiao G, Liu G (2017). Multi-sensor image fusion based on fourth order partial differential equations. In: 2017 20th International conference on information fusion (Fusion) (pp. 1–9). IEEE
Broussard RP, Rogers SK, Oxley ME, Tarr GL (1999) Physiologically motivated image fusion for object detection using a pulse coupled neural network. IEEE Trans Neural Netw 10(3):554–563
Deshmukh M, Bhosale U (2010) Image fusion and image quality assessment of fused images. Int J Image Proces (IJIP) 4(5):484
Dogra A, Goyal B, Agrawal S (2017) From multi-scale decomposition to non-multi-scale decomposition methods: a comprehensive survey of image fusion techniques and its applications. IEEE Access 5:16040–16067
Eskicioglu AM, Fisher PS (1995) Image quality measures and their performance. IEEE Trans Commun 43(12):2959–2965
Guo Y, Li C, Liu Q (2019) R ~ 2n: a novel deep learning architecture for rain removal from single image. Comput Mater Continua 58(3):829–843
James AP, Dasarathy BV (2014) Medical image fusion: a survey of the state of the art. Inf Fusion 19:4–19
Lewis JJ, O’Callaghan RJ, Nikolov SG, Bull DR, Canagarajah N (2007) Pixel-and region-based image fusion with complex wavelets. Inf Fusion 8(2):119–130
Li H, Wu XJ (2018) DenseFuse: a fusion approach to infrared and visible images. IEEE Trans Image Process 28(5):2614–2623
Li S, Kang X, Hu J (2013) Image fusion with guided filtering. IEEE Trans Image Process 22(7):2864–2875
Li S, Kang X, Fang L, Hu J, Yin H (2017) Pixel-level image fusion: a survey of the state of the art. Inf Fusion 33:100–112
Li Y, Sun Y, Huang X, Qi G, Zheng M, Zhu Z (2018) An image fusion method based on sparse representation and sum modified-Laplacian in NSCT domain. Entropy 20(7):522
Liu Y, Chen X, Peng H, Wang Z (2017) Multi-focus image fusion with a deep convolutional neural network. Inf Fusion 36:191–207
Liu X, Mei W, Du H (2018) Multi-modality medical image fusion based on image decomposition framework and nonsubsampled shearlet transform. Biomed Signal Process Control 40:343–350
Liu Z, Xiang B, Song Y, Lu H, Liu Q (2019) An improved unsupervised image segmentation method based on multi-objective particle swarm optimization clustering algorithm. Comput Mater Continua 58(2):451–461
Lu GX, Zhou DW, Wang JL (2002) Geological information extracting from remote sensing image in complex area: based on wavelet analysis for automatic image segmentation. Earth Science-Journal of China University of Geosciences 27(1):50–54
Ma J, Chen C, Li C, Huang J (2016a) Infrared and visible image fusion via gradient transfer and total variation minimization. Inf Fusion 31:100–109
Ma Y, Chen J, Chen C, Fan F, Ma J (2016b) Infrared and visible image fusion using total variation model. Neurocomputing 202:12–19
Ma J, Zhou Z, Wang B, Zong H (2017) Infrared and visible image fusion based on visual saliency map and weighted least square optimization. Infrared Phys Technol 82:8–17
Ma J, Yu W, Liang P, Li C, Jiang J (2019) FusionGAN: a generative adversarial network for infrared and visible image fusion. Information Fusion 48:11–26
Mohanapriya N, Kalaavathi B (2019) Adaptive image enhancement using hybrid particle swarm optimization and watershed segmentation. Intell Autom soft comput 25(4):663–672
Nejad MB, Shiri ME (2019) A new enhanced learning approach to automatic image classification based on salp swarm algorithm. Int J Comput Syst Sci Eng 34(2):91–100
Pan L, Qin J, Chen H, Xiang X, Li C, Chen R (2019) Image augmentation-based food recognition with convolutional neural networks. Comput Mater Continua 59(1):297–313
Qin J, Huang Y, Wen W (2020) Multi-scale feature fusion residual network for single image super-resolution. Neurocomputing 379:334–342
Ram Prabhakar K, Sai Srikar V, Venkatesh Babu R (2017) DeepFuse: a deep unsupervised approach for exposure fusion with extreme exposure image pairs. In: Proceedings of the IEEE international conference on computer vision (pp. 4714-4722)
Shachor Y, Greenspa H, Goldberger J (2019) A mixture of views network with applications to the classification of breast microcalcifications. In: 2019 IEEE 16th International symposium on biomedical imaging (ISBI 2019) (pp. 1065–1069). IEEE
Wang J, Peng J, Feng X, He G, Fan J (2014) Fusion method for infrared and visible images by using non-negative sparse representation. Infrared Phys Technol 67:477–489
Wang Q, Lai J, Yang Z, Xu K, Kan P, Liu W, Lei L (2019) Improving cross-dimensional weighting pooling with multi-scale feature fusion for image retrieval. Neurocomputing 363:17–26
Wu L, Liu Q, Lou P (2019) Image classification using optimized mkl for sspm. Intell Autom Soft Comput 25(2):249–257
Yin M, Liu X, Liu Y, Chen X (2018) Medical image fusion with parameter-adaptive pulse coupled neural network in nonsubsampled shearlet transform domain. IEEE Trans Instrum Meas 68(1):49–64
Zhang X, Ma Y, Fan F, Zhang Y, Huang J (2017) Infrared and visible image fusion via saliency analysis and local edge-preserving multi-scale decomposition. JOSA A 34(8):1400–1410
Zhao J, Cui G, Gong X, Zang Y, Tao S, Wang D (2017) Fusion of visible and infrared images using global entropy and gradient constrained regularization. Infrared Phys Technol 81:201–209
Zhu Z, Zheng M, Qi G, Wang D, Xiang Y (2019) A phase congruency and local Laplacian energy based multi-modality medical image fusion method in NSCT domain. IEEE Access 7:20811–20824
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that there is no conflict of interest with any person(s) or organization(s).
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Pan, Y., Pi, D., Khan, I. et al. DenseNetFuse: a study of deep unsupervised DenseNet to infrared and visual image fusion. J Ambient Intell Human Comput 12, 10339–10351 (2021). https://doi.org/10.1007/s12652-020-02820-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-020-02820-3