Abstract
Multi-Focus Image Fusion (MFIF) is a method that combines two or more source images to obtain a single image which is focused, has improved quality and more information than the source images. Due to limited Depth-of-Field of the imagining system, extracting all the useful information from a single image is challenging. Thus two or more defocused source images are fused together to obtain a composite image. This paper provides a comprehensive overview of existing MFIF methods. A new classification scheme is developed for categorizing the existing MFIF methods. These methods are classified into four major categories: spatial domain, transform domain, deep leaning and their hybrids and have been discussed well along with their drawbacks and challenges. In addition to this, both the parametric evaluation metrics i.e. "with reference" and "without reference" have also discussed. Then, a comparative analysis for nine image fusion methods is performed based on 30 pairs of publicly available images. Finally, various challenges that remain unaddressed and future work is also discussed in this work.






Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Agrawal D, Singhai J (2010) Multifocus image fusion using modified pulse coupled neural network for improved image quality. IET Image Proc 4(6):443–451
Aslantas V, Kurban R (2009) A comparison of criterion functions for fusion of multi-focus noisy images. Opt Commun 282(16):3231–3242
Aslantas V, Kurban R (2010) Fusion of multi-focus images using differential evolution algorithm. Expert Syst Appl 37(12):8861–8870
Aymaz S, Kose C, (2017) Multi-focus image fusion using Stationary Wavelet Transform (SWT) with Principal Component Analysis (PCA): In Proceedings of the 10th international conference on electrical and electronics engineering (ELECO). pp 1176–1180
Aymaz S, Kose C (2019) A novel image decomposition-based hybrid technique with super-resolution method for multi-focus image fusion. Inf Fus 45:113–127
Bai X, Zhang Y, Zhou F, Xue B (2015) Quadtree-based multi-focus image fusion using a weighted focus-measure. Inf Fus 22(1):105–118
Bai X, Liu M, Chen Z, Wang P, Zhang Y (2016) Multi-focus image fusion through gradient-based decision map construction and mathematical morphology. IEEE Access 4:4749–4760
Balasubramaniam P, Ananthi VP (2014) Image fusion using intuitionistic fuzzy sets. Inf Fus 20:21–30
Banharnsakun A (2019) Multi-focus image fusion using best-so-far abc strategies. Neural Comput Appl 31(7):2025–2040
Bavirisetti DP, Dhuli R (2016) Fusion of infrared and visible sensor images based on anisotropic diffusion and karhunen-loeve transform. IEEE Sens J 16(1):203
Bhandari PK, Venkasteshappa CPR (2016) An efficient method of image fusion using SWT & DTCWT. Int J Electron Commun Eng 9(1):29–37
Bhat S, Koundal D (2019) Multi-focus image fusion: quantitative and qualitative comparative analysis. In: Proceedings of the 2nd international conference on recent innovations in computing (ICRIC). pp 533–542
Bogoni L, Hansen M (2001) Pattern-selective color image fusion. Pattern Recogn 34:1515–1526
Bouzos O, Andreadis I, Mitianoudis N (2019) Conditional random field model for robust multi-focus image fusion. IEEE Trans Image Process 28(11):5636–5648
Burt P, Adelson E (1985) Merging images through pattern decomposition. Appl Digital Image Process VIII 575:173–181
Burt P, Kolcznski R (1993) Enhanced image capture through fusion. In: Proceedings of IEEE Interational Coference on Computer Vision (ICCV). pp 173–182
Chai Y, Li H, Zhang X (2012b) Multifocus image fusion based on features contrast of multiscale products in nonsubsampled contourlet transform domain. Optik Int J Light Electron Opt 123(7):569–581
Chai Y, Li H, Zhang X (2012a) Multifocus image fusion based on features contrast of multiscale products in nonsubsampled contourlet transform domain. Optik Int J Light Electron Opt 123(7):569–581
Chen Y, Guan J, Cham WK (2007) Robust multi-focus image fusion using edge model and multi-matting. IEEE Trans Image Process 27(3):1526–1541
Chen L, Li J, Chen C (2013) Regional multifocus image fusion using sparse representation. Opt Express 21(4):5182–5197
Chen Z, Wang D, Gong S, Zhao F (2017) Application of multi-focus image fusion in visual power patrol inspection. Paper presented at 2nd advanced information technology electronic and automation control conference (IAEAC). pp 1688–1692
da Cunha AL, Zhou J, Do MN (2006) The nonsubsampled contourlet transform: theory, design, and applications. IEEE Trans Image Process 15(10):3089–3101
Du C, Gao S (2018) Multi-focus image fusion algorithm based on pulse coupled neural networks and modified decision map. Optik 157:1003–1015
Easley G, Labate D, Lim W (2008) Sparse directional image representations using the discrete shearlet transform. Appl Comput Harmon Anal 25:25–46
Farid MS, Mahmood A, Al-Maadeed SA (2019) Multi-focus image fusion using content adaptive blurring. Inf Fus 45:96–112
Garg R, Gupta P, Kaur H, (2014) Survey on multi-focus image fusion algorithms. Paper presented on Recent Advances in Engineering and Computational Sciences (RAECS). pp. 1–5
Guo L, Dai M, Zhu M (2012) Multifocus color image fusion based on quaternion curvelet transform. Opt Express 20(17):18846–18860
Guo X, Nie R, Cao J, Zhou D, Qian W (2018) Fully convolutional network-based multifocus image fusion. Neural Comput 30(7):1775–1800
Haghighat MBA, Aghagolzadeh A, Seyedarabi H (2010) Real-time fusion of multi-focus images for visual sensor networks. Paper presented at 6th Iranian conference on machine vision and image processing. pp 1–6
Haghighat MBA, Aghagolzadeh A, Seyedarabi H (2011) Multi-focus image fusion for visual sensor networks in DCT domain. Comput Electr Eng 37(5):789–797
Hao X, Zhao H, Liu J (2015) Multifocus color image sequence fusion based on mean shift segmentation. Appl Opt 54(30):8982–8989
He K, Zhou D, Zhang X, Nie R, Jin X (2018) Multi-focus image fusion combining focus-region-level partition and pulse-coupled neural network. Soft Comput 23:1–15
Helonde MR, Joshi MR (2015) Image fusion based on medical images using DWT and PCA methods. Int J Comput Tech 2(1):75–79
Hong R, Wang C, Ge Y, Wang M, Wu X, Zhang R (2007) Salience preserving multi-focus image fusion. In: Proceedings of IEEE international conference on multimedia and expo (ICME). pp 1663–1666
Hou R, Zhou D, Nie R, Liu D (2018) Multi-focus color image fusion scheme using NSST and focus region detection. In: Proceedings of the 3rd international conference on multimedia and image processing. pp 7–11
Hu K, Ye J, Fan E, Shen S, Huang L, Pi J (2017) A novel object tracking algorithm by fusing color and depth information based on single valued neutrosophic cross-entropy. J Intell Fuzzy Syst 32(3):1775–1786
Hua KL, Wang HC, Rusdi AH, Jiang SY (2014) A novel multi-focus image fusion algorithm based on random walks. J Vis Commun Image Represent 25(5):951–962
Huang W, Jing Z (2007) Multi-focus image fusion using pulse coupled neural network. Pattern Recognit Lett 28(9):1123–1132
Huang Y, Li W, Gao M, Liu Z (2018) Algebraic multi-grid based multi-focus image fusion using watershed algorithm. IEEE Access 6:47802–47091
Ji Z, Kang X, Zhang K, Duan P, Hao Q (2020) A two-stage multi-focus image fusion framework robust to image mis-registration. IEEE Access 7:123231–123243
Jiang ZG, Han DB, Chen J, Zhou XK (2004) A wavelet based algorithm for multi-focus micro-image fusion. In: Proceedings of the third international conference on image and graphics (ICIG). pp 176–179
Jiang Q, Jin X, Lee SJ, Yao S (2017) A novel multi-focus image fusion method based on stationary wavelet transform and local features of fuzzy sets. IEEE Access 5:20286–20302
Jin X, Hou J, Nie R, Yao S, Zhou D, Jiang Q, He K (2018) A lightweight scheme for multi-focus image fusion. Multimed Tools Appl 77(18):20286–20302
K. Sujatha K, D.S. Punithavathani, (2018) Optimized ensemble decision-based multi-focus image fusion using binary genetic Grey-Wolf optimizer in camera sensor networks. Multimed Tools Appl 77(2):1735–1759
Kannan K, Perumal AS, Arulmozhi K (2010) Optimal decomposition level of discrete, stationary and dual tree complex wavelet transform for pixel based fusion of multi-focused images. Serbian J Electr Eng 7(1):81–93
Kaur G, Kaur P (2016) Survey on multifocus image fusion techniques. Paper presented at international conference on electrical, electronics, and optimization techniques (ICEEOT). pp 1420–1424
Kaur H, Rani EJ (2015) Analytical comparison of various image fusion techniques. Int J Adv Res Comput Sci Softw Eng 5(5):1–5
Kaur H, Koundal D, Kadyan V (2021) Image fusion techniques: a survey. Arch Computat Methods Eng. https://doi.org/10.1007/s11831-021-09540-7
Kong J, Zheng K, Zhang J, Feng X (2008) Multi-focus image fusion using spatial frequency and genetic algorithm. Int J Comput Sci Netw Secur 8(2):220–224
Kou L, Zhang L, Zhang K, Sun J, Han Q, Jin Z (2018) A multi-focus image fusion method via region mosaicking on Laplacian pyramids. PLoS ONE 13(5):0191085
Lai R, Li Y, Guan J, Xiong A (2019) Multi-scale visual attention deep convolutional neural network for multi-focus image fusion. IEEE Access 7:114385–114399
Li H, Wu XJ, (2018) Multi-focus noisy image fusion using low-rank representation, arXiv preprint arXiv: 1804.09325
Li S, Yang B (2008) Multifocus image fusion using region segmentation and spatial frequency. Image Vis Comput 26(7):971–979
Li S, Kwok J, Wang Y (2001) Combination of images with diverse focuses using the spatial frequency. Inf Fus 2(3):169–176
Li S, Kwok JT, Wang Y (2002) Multifocus image fusion using artificial neural networks. Pattern Recogn Lett 23(8):985–997
Li Z, Jing Z, Liu G, Sun S, Leung H (2003) Pixel visibility based multifocus image fusion. Paper presented at international conference on neural networks and signal processing. pp 1050–1053
Li M, Cai W, Tan Z (2006) A region-based multi-sensor image fusion scheme using pulse-coupled neural network. Pattern Recogn Lett 27(16):1948–1956
Li H, Chai Y, Li Z (2013) A new fusion scheme for multifocus images based on focused pixels detection. Mach Vis Appl 24(6):1167–1181
Li H, Liu X, Yu Z, Zhang Y (2016) Performance improvement scheme of multifocus image fusion derived by difference images. Signal Process 128:474–493
Li H, Qiu H, Yu Z, Li B (2017) Multifocus image fusion via fixed window technique of multiscale images and non-local means filtering. Signal Process 138:71–85
Li H, Nie R, Cao J, Guo X, Zhou D, He K (2019) Multi-focus image fusion using u-shaped networks with a hybrid objective. IEEE Sens J 19(21):9755–9765
Li J, Guo X, Lu G, Zhang B, Xu Y, Wu F, Zhang D (2020) Drpl: deep regression pair learning for multi-focus image fusion. IEEE Trans Image Process 29:4816–4831
Liu Y, Wang Z (2015) Simultaneous image fusion and denosing with adaptive sparse representation. IET Image Proc 9(5):347–357
Liu Y, Jin J, Wang Q, Shen Y, Dong X (2013) Novel focus region detection method for multifocus image fusion using quaternion wavelet. J Electron Imag 22(2):023017-1-023017–17
Liu Y, Jin J, Wang Q, Shen Y, Dong X (2014) Region level based multi-focus image fusion using quaternion wavelet and normalized cut. Signal Process 97:9–30
Liu Y, Liu S, Wang Z (2015) Multi-focus image fusion with dense SIFT. Inf Fus 23(1):139–155
Liu S, Shi M, Zhu Z, Zhao J (2017b) Image fusion based on complex-shearlet domain with guided filtering. Multidimens Syst Signal Process 28:207–224
Liu Y, Chen X, Peng H, Wang Z (2017a) Multi-focus image fusion with a deep convolutional neural network. Inf Fus 36:191–207
Liu Y, Wang L, Cheng J, Li C, Chen X (2020) Multi-focus image fusion: a survey of the state of the art. Inf Fus 64:71–91
Lu Y, Feng X, Zhang J, Wang R, Zheng K, Kong J (2007) A multi-focus image fusion based on wavelet and region detection. Paper presented at international conference on computer as a tool. pp 294–298
Ludusan C, Lavialle O (2012) Multifocus image fusion and denoising: a variational ap- proach. Pattern Recognit Lett 33(10):1388–1396
Ma X, Hu S, Liu S, Fang J, Xu S (2019b) Multi-focus image fusion based on joint sparse representation and optimum theory. Signal Process Image Commun 78:125–134
Ma J, Zhou Z, Wang B, Miao L, Zong H (2019a) Multi-focus image fusion using boosted random walks-based algorithm with two-scale focus maps. Neurocomputing 335:9–20
Miao Q, Wang B (2005) A novel adaptive multi-focus image fusion algorithm based on PCNN and sharpness. Sens Command Control Commun Intell Technol Homeland Security Homeland Defense 5778:704–713
Miao Q, Shi C, Xu P, Yang M, Shi Y (2011) A novel algorithm of image fusion using shearlets. Opt Commun 284(6):1540–1547
Ming L, Yan W, Shunjun W (2003) Multi-focus image fusion based on wavelet decomposition and evolutionary strategy. Paper presented at international conference on neural networks and signal processing. pp 951–955
Ming L, Yan W, Shunjun W (2004) A new pixel-level multi-focus image fusion algorithm based on evolutionary strategy. Paper presented at conference on control, automation, robotics and vision. pp 810–814
Naidu V (2013) Novel image fusion techniques using dct. Int J Comput Sci Bus Inf 5(1):1–18
Nejati M, Samavi S, Shirani S (2015) Multi-focus image fusion using dictionary-based sparse representation. Inf Fus 25(1):72–84
Nejati M, Samavi S, Karimi N, Soroushmehr SR, Shirani S, Roosta I, Najarian K (2017) Surface area-based focus criterion for multi-focus image fusion. Inf Fus 36:284–295
Pajares G, Cruz JM (2004) A wavelet-based image fusion tutorial. Pattern Recogn 37(9):1855–1872
Patel R, Rajput M, Parekh P (2015) Comparative study on multi-focus image fusion techniques in dynamic scene. Int J Comput Appl 109:5–9
Paul S, Sevcenco I, Agathoklis P (2016) Multi-exposure and multi-focus image fusion in gradient domain. J Circuits Syst Comput 25(10):1650123
Phamila YA, Amutha R (2014) Discrete cosine transform based fusion of multi-focus images for visual sensor networks. Signal Process 95:161–170
Piella G (2009) Image fusion for enhanced visualization: avariational approach. Int J Comput Vis 83(1):1–11
Plas RVD, Yang J, Spraggins J, Caprioli RM (2015) Image fusion of mass spectrometry and microscopy: a multimodality paradigm for molecular tissue mapping. Nat Methods 12(4):366–372
Qayyum H, Majid M, Anwar SM, Khan B (2017) Facial expression recognition using stationary wavelet transform features. Math Probl Eng 2017:1–9
Saeedi J, Faez K, Mozaffari S (2009) Multi-focus image fusion based on fuzzy and wavelet transform. Iberoamerican Congress on Pattern Recognition. pp 970–977
Sahu DK, Parsai MP (2012) Different image fusion techniques–a critical review. Int J Modern Eng Res 2(5):4298–4301
Siddiqui AB, Rashid M, Jaffar MA, Hussain A, Mirza AM (2011) Feature classification for multi-focus image fusion. Int J Phys Sci 6(20):4838–4847
Simone G, Farina A, Morabito FC, Serpico SB, Bruzzone L (2002) Image fusion techniques for remote sensing applications. Inf Fus 3(1):3–15
Singh T, Manchanda P (2018) Multi-focus image fusion using modified gaussian filter and discrete lifting wavelet transform. Int J Electron Eng 10(2):201–209
Singh S, Patil MM (2016) Multi focus image fusion based on spatial frequency and contrast based analysis under stationary wavelet transform domain. Int J Sci Eng Res 7(5):225–230
Sivagami R, Vaithiyanathan V, Sangeetha V, Ahmed MI, Sundar KJA, Lakshmi KD (2015) Review of image fusion techniques and evaluation metrics for remote sensing applications. Indian J Sci Technol 8:1–7
Sun J, Zhu H, Xu Z, Han C (2013) Poisson image fusion based on markov random field fusion model. Inf Fus 14(3):241–254
Sun J, Han Q, Kou L, Zhang L, Zhang K, Jin Z (2018) Multi-focus image fusion algorithm based on Laplacian pyramids. JOSA A 35(3):480–490
Tang H, Xiao B, Li W, Wang G (2018) Pixel convolutional neural network for multi-focus image fusion. Inf Sci 433(1):125–141
Tian J, Chen L (2012) Adaptive multi-focus image fusion using a wavelet-based statistical sharpness measure. Signal Process 92(9):2137–2146
Vakaimalar E, Mala K, Babu RS (2019) Multifocus image fusion scheme based on discrete cosine transform and spatial frequency. Multimed Tools Appl 78(13):17573–17587
Wan T, Zhu C, Qin Z (2013) Multifocus image fusion based on robust principal component analysis. Pattern Recogn Lett 34(9):1001–1008
Wang Z, Ma Y (2008) “Medical image fusion using m-PCNN. Inf Fus 9(2):176–185
Wei S, Ke W (2007) A multi-focus image fusion algorithm with DT-CWT. Paper presented at international conference on computationaal intelligence and security. pp 147–151
Xia X, Yao Y, Yin L, Wu S, Li H, Yang Z (2018) Multi-focus image fusion based on probability filtering and region correction. Signal Process 153:71–82
Xu K, Qin Z, Wang G, Zhang H, Huang K, Ye S (2018) Multi-focus image fusion using fully convolutional two-stream network for visual sensors. KSII Trans Internet Inf Syst 12(5):2253–2272
Song Y, Li M, Li Q, Sun L (2006) A new wavelet based multi-focus image fusion scheme and its application on optical microscopy. Paper presented at international conference on robotics and biomimetics (ROBIO).pp 401–405
Yang Y (2011) A novel DWT based multi-focus image fusion method. Int Conf Adv Eng 24:177–181
Yang B, Li S (2010) Multifocus image fusion and restoration with sparse representation. IEEE Trans Instrum Meas 59(4):884–892
Yang B, Li S (2012) Pixel-level image fusion with simultaneous orthogonal matching pursuit. Inf Fus 13(1):10–19
Yang B, Li S, Sun F, (2007) Image fusion using nonsubsampled contourlet transform. In: Proceedings of international conference on image and graphics (ICIG). pp 719–724
Yang Y, Zheng W, Huang S (2014) Effective multifocus image fusion based on HVS and BP neural network. Sci World J 14(1):1–10
Yang Y, Yang M, Huang S, Ding M, Sun J (2018) Robust sparse representation combined with adaptive PCNN for multifocus image fusion. IEEE Access 6:20138–201351
Yang Y, Nie Z, Huang S, Lin P, Wu J (2019) Multilevel features convolutional neural network for multifocus image fusion. IEEE Trans Comput Imaging 5(2):262–273
Yin H, Li Y, Chai Y, Liu Z, Zhu Z (2016) A novel sparse-representation-based multi-focus image fusion approach. Neurocomputing 216(5):216–229
Zafar I, Edirisinghe EA, Bez HE (2006) Multi-exposure & multi-focus image fusion in transform domain. In: IET international conference on visual information engineering. pp 606–611
Zhang Q, Guo B (2009) Multifocus image fusion using the nonsubsampled contourlet transform. Signal Process 89(7):1334–1346
Zhang B, Zhang C, Yuanyuan L, Jianshuai W, He L (2014) Multi-focus image fusion algorithm based on compound pcnn in surfacelet domain. Optik (Stuttg) 125(1):296–300
Zhang Y, Liu Y, Sun P, Yan H, Zhao X, Zhang L (2020) Ifcnn: a general image fusion framework based on convolutional neural network. Inf Fus 54:99–118
Zhao W, Wang D, Lu H (2018) Multi-focus image fusion with a natural enhancement via joint multi-level deeply supervised convolutional neural network. IEEE Transact Circuit Syst Video Technol 29(4):1102–1115
Zhou Z et al (2016a) Fusion of infrared and visible images for night-vision context enhancement. Appl Opt 55(23):6480
Zhou Z et al (2016b) Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters. Inf Fus 30:15–26
Zhou Z, Li S, Wang B (2014) Multi-scale weighted gradient-based fusion for multi-focus images. Inf Fus 20:60–72
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bhat, S., Koundal, D. Multi-focus image fusion techniques: a survey. Artif Intell Rev 54, 5735–5787 (2021). https://doi.org/10.1007/s10462-021-09961-7
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10462-021-09961-7