Abstract
Image fusion aims to integrate multiple images of the same scene into an artificial image which contains more useful information than any individual one. Due to the constraints of imaging sensors and signal transmission broadband, the resolution of most source images is limited. In traditional processing framework, super-resolution is conducted to improve the resolution of the source images before the fusion operations. However, those super-resolution methods do not make full use of the multi-resolution characteristics of images. In this paper, a novel jointed image fusion and super-resolution algorithm is proposed. Source images are decomposed into undecimated wavelet (UWT) coefficients, the resolution of which is enhanced with convolutional neural network. Then, the coefficients are further integrated with certain fusion rule. Finally, the fused image is constructed from the combined coefficients. The proposed method is tested on multi-focus images, medical images and visible light and near infrared ray images respectively. The experimental results demonstrate the superior performances of the proposed method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Li, S.T., Kang, X.D., Hu, J.W.: Image fusion with guided filtering. J. IEEE Trans. Image Process. 22, 2864–2875 (2013)
Li, H., Li, L., Zhang, J.X.: Multi-focus image fusion based on sparse feature matrix decomposition and morphological filtering. J. Opt. Commun. 342, 1–11 (2015)
Chai, Y., Li, H.F., Guo, M.Y.: Multifocus image fusion scheme based on features of multiscale products and PCNN in lifting stationary wavelet domain. J. Opt. Commun. 284, 1146–1158 (2011)
Mirajkar, P.P., Sachin, D.R.: Image fusion based stationary wavelet transform. J. Int. J. Adv. Eng. Res. Stud. 2, 99–101 (2013)
Lewis, J.J., Callaghan, R.J., Nikolov, S.G.: Pixel- and region-based image fusion with complex wavelets. J. Inf. Fusion 8, 119–130 (2007)
Sravya, K., Govardhan, P., Goud, M.N.: Image fusion on multi focused images using NSCT. J. IJCSIT 5, 5393–5396 (2014)
Lui, S.-F., Wu, J.-Y., Mao, H.-S., Lien, J.-J.J.: Learning-based super-resolution system using single facial image and multi-resolution wavelet synthesis. In: Yagi, Y., Kang, S.B., Kweon, I.S., Zha, H. (eds.) ACCV 2007, Part II. LNCS, vol. 4844, pp. 96–105. Springer, Heidelberg (2007)
Liu, W.R., Li, S.T.: Image super-resolution based on hybrid multiresolution analysis. In: The First Asian Conference on Pattern Recognition, Beijing, China, pp. 307–310. IEEE press (2011)
Zhao, G., Zhang, K., Shao, W.: A study on NSCT based super-resolution reconstruction for infrared image. In: TENCON 2013 – 2013 IEEE Region 10 Conference, Xi’an, China, pp. 1–5. IEEE press (2013)
Lu, X., Li, X.: Multiresolution imaging. J. IEEE Trans. Cybern. 44, 149–160 (2014)
Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part IV. LNCS, vol. 8692, pp. 184–199. Springer, Heidelberg (2014)
Yang, J., Wright, J., Huang, T.S.: Image super-resolution via sparse representation. J. IEEE Trans. Image Process. 19, 2861–2873 (2010)
Ren, J.S., Xu, L., Yan, Q.: Shepard convolutional neural networks. In: Proceeding of 28th Annual Conference on Neural Information Processing Systems, Montreal, Canada. MIT press (2015)
Bevilacqua, M., Roumy, A., Guillemot, C.: Low-complexity single-image super-resolutionbased on nonnegative neighbor embedding. In: Proceedings of the 23rd British Machine Vision Conference (BMVC), pp. 135.1–135.10. BMVC press, Guidford (2012)
Chang, H., Yeung, D.Y., Xiong, Y.: Super-resolution through neighbor embedding. In: IEEE Conference on Computer Vision and Pattern Recognition, California, USA, pp. 275–282. IEEE Computer Society (2010)
Jia, K., Wang, X., Tang, X.: Image transformation based on learning dictionaries across image spaces. J. IEEE Trans. Pattern Anal. Mach. Intell. 35, 367–380 (2013)
Timofte, R., De, V., Van Gool, L.: Anchored neighborhood regression for fast example-based super-resolution. In: IEEE International Conference on Computer Vision, Sydney, Australia, pp. 1920–1927. IEEE press (2013)
Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Boissonnat, J.-D., Chenin, P., Cohen, A., Gout, C., Lyche, T., Mazure, M.-L., Schumaker, L. (eds.) Curves and Surfaces 2011. LNCS, vol. 6920, pp. 711–730. Springer, Heidelberg (2012)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. J. Nat. 323, 533–536 (1986)
Piella, G., Heijmans, H.: A new quality metric for image fusion. In: Proceedings of the IEEE International Conference on Image Processing, Barcelona, Spain, pp. 173–176. IEEE press (2003)
Xydeas, C.S., Petrovic, V.: Objective image fusion performance measure. J. Electron. Lett. 36, 308–309 (2000)
Acknowledgments
This paper is supported by the National Natural Science Foundation of China (Nos. 61102108), Scientific Research Fund of Hunan Provincial Education Department (Nos. YB2013B039), Young talents program of the University of South China, and the construct program of key disciplines in USC (No. NHXK04).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zhong, J., Yang, B., Li, Y., Zhong, F., Chen, Z. (2016). Image Fusion and Super-Resolution with Convolutional Neural Network. In: Tan, T., Li, X., Chen, X., Zhou, J., Yang, J., Cheng, H. (eds) Pattern Recognition. CCPR 2016. Communications in Computer and Information Science, vol 663. Springer, Singapore. https://doi.org/10.1007/978-981-10-3005-5_7
Download citation
DOI: https://doi.org/10.1007/978-981-10-3005-5_7
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-3004-8
Online ISBN: 978-981-10-3005-5
eBook Packages: Computer ScienceComputer Science (R0)