Abstract
Image fusion is a process of combing multiple images of the same scene into a single image with the aim of preserving the full content information and retaining the important features from each of the original images. In this paper, a novel image fusion method based on Convolutional Neural Networks (CNN) and saliency detection is proposed. Here, we use the image representations derived from CNN Network optimized for infrared-visible image fusion. Since the lower layers of the network can seize the exact value of the original image, and the high layers of the network can capture the high-level content in terms of objects and their arrangement in the input image, we exploit more low-layer features of visible image and more high-layer features of infrared image in the fusion. And during the fusion procedure, the infrared target of an infrared image is effectively highlighted using saliency detection method and only the salient information of the infrared image will be fused. The method aimed to preserve the abundant detail information from visible image as much as possible, meanwhile preserve the salient information in the infrared image. Experimental results show that the proposed fusion method is rather promising.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bavirisetti, D.P., Dhuli, R.: Fusion of infrared and visible sensor images based on anisotropic diffusion and Karhunen-Loeve transform. IEEE Sens. J. 16(1), 203–209 (2016)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: International Conference on Neural Information Processing Systems, pp. 1097–1105 (2012)
Gatys, L.A., Ecker, A.S.: Image style transfer using convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2414–2423. IEEE (2016)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. Comput. Sci. (2014)
Li, H., Wu, X., Kittler, J.: Infrared and Visible Image Fusion using a Deep Learning Framework. arXiv:1804.06992 [cs.CV] (2018)
Mahendran, A., Vedaldi, A.: Understanding deep image representation by inverting them. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5188–5196. IEEE (2015)
Gatys, L.A., Ecker, A.S., Bethge, M.: Texture synthesis using convolutional neural networks. In: Advances in Neural Information Processing Systems, vol. 70, no. 1, pp. 262–270 (2015)
Zhu, C., Byrd, R.H., Lu, P., Nocedal, J.: Algorithm 778: L-BFGS-B: fortran subroutine for large-scale bound-constrained optimization. ACM Trans. Math. Softw. (TOMS) 23(4), 550–560 (1997)
Acknowledgments
This paper is supported by the National Natural Science Foundation of China (NSFC. No. 61271420), Scientific Research Platform Cultivation Project of SZIIT (PT201704), Scientific Research Project of SZIIT (ZY201715).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Ren, X., Meng, F., Hu, T., Liu, Z., Wang, C. (2018). Infrared-Visible Image Fusion Based on Convolutional Neural Networks (CNN). In: Peng, Y., Yu, K., Lu, J., Jiang, X. (eds) Intelligence Science and Big Data Engineering. IScIDE 2018. Lecture Notes in Computer Science(), vol 11266. Springer, Cham. https://doi.org/10.1007/978-3-030-02698-1_26
Download citation
DOI: https://doi.org/10.1007/978-3-030-02698-1_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-02697-4
Online ISBN: 978-3-030-02698-1
eBook Packages: Computer ScienceComputer Science (R0)