Elsevier

Applied Soft Computing

Volume 51, February 2017, Pages 314-327
Applied Soft Computing

Multi-focus image fusion using biochemical ion exchange model

https://doi.org/10.1016/j.asoc.2016.11.033Get rights and content

Highlights

  • Mineral nutrient uptake mechanism is firstly used to in the area of image fusion.

  • The pixel in each image is viewed as the ion with different valence and polarity.

  • The algorithm of the biochemical ion exchange model is devised.

  • The proposed technique has obvious superiorities.

Abstract

An effective and efficient multi-focus image fusion technique is presented for creating a more informative composite image for further human or machine perceptions. The proposed technique is based on a novel theory called biochemical ion exchange model (BIEM) which stems from the mechanism of the mineral nutrient uptake mechanism of the root. The source images with different focuses are considered to be the inputs of BIEM which is proposed in this paper, respectively, and the pixels can be viewed as the ions with different valences and polarities. The spatial frequency is chosen as the determinant of the valence. The final fused image can be obtained under the influence of BIEM and the new SSIM metric. Experimental results and relevant analysis indicate that the proposed fusion technique is promising and has remarked superiorities over other current typical fusion ones.

Graphical abstract

As descried in GA 1, the root has the ability to get the ions needed by exchanging its own ions with the ones in the soil. By careful comparison between the fusion process and the cellular respiration, we have sufficient reasons to believe that these two processes virtually have something in common.

Multi-focus source images belonging to the same group describe the same scene, but the imaging sensors cannot put all objects in different distances in focus, as the root is unable to obtain all nutrients via cellular respiration. Therefore, the fusion of multi-focus images is necessary, and the fused image is obtained by selecting the pixels in focus from different source images and then fusing them in a single one in essence.

By comparing the fusion process and the cellular respiration, we can find that so many similarities exist in terms of the principle. In this section, we have described the fusion framework of multi-focus images based on BIEM. The fusion course can be carried out via the framework. For simplicity, we take a pair of multi-focus source images denoted by A and B respectively for example. It is noteworthy that A is regarded as the benchmark, and the source images should be strictly registered to ensure the corresponding pixels have been aligned. The schematic diagram of the proposed fusion framework is depicted in GA 2. GA 2 consists of three main phases including region determination, region selection, and iteration, which are detailed presented as follows.

  1. Download : Download high-res image (118KB)
  2. Download : Download full-size image

Introduction

Image fusion is a technology to extract the complementary or redundant features from the given individual source images of the same scene and then fuse them into a composite image with more dependent and more comprehensive information than for further human or machine perceptions. With several decades’ development of the imaging technology, image fusion has attracted increasing attentions all over the world and has been widely used in a great many of areas, such as medical imaging [1] and remote sensing [2].

According to the distinction of the imaging principle, the most common types of source images can be mainly classified into homogeneous and heterogeneous categories, the fundamental difference of which lies in whether the group of images are from the same pattern of imaging sensors or not. Obviously, multi-focus source images are a representative homogeneous example, whereas heterogeneous imaging sensors often produce the images with entirely different imaging purposes, e.g., gray-scale and infrared images, computed tomography and magnetic resonance imaging images.

Since optical imaging cameras often suffer from the predicament of finite depth-of-focus lens, they cannot put all objects in different distances in focus. The corresponding vision effects are that several objects are in focus, while other ones of the same scene may exactly out of focus and, thus, blurred [3]. So far, extensive researches and various techniques [4], [5], [6], [7], [8], [9], [10], [11] have been done and developed dedicated to the fusion field of multi-focus images. Basically, these techniques can be mainly categorized into the following two categories in terms of the fusion strategy. The first category is based on the spatial domain transform (SDT). A well-known intuitive one in SDT is the weighted technique (WT) [12], and the final fused image is estimated as the weighted compromise among the pixels with the same spatial location in the corresponding inputs. WT is resistant to the existent noise in the inputs to a certain degree, but it always results in the decline of the contrast level of the fused image. Furthermore, the theories of principal component analysis (PCA) [13] and independent component analysis (ICA) [14] have been used for image fusion as well. However, the methods based on PCA and ICA both put forward high requirements to the component selection. Recently, as the third generation of the artificial neural networks (ANN), pulse coupled neural network (PCNN) [15] as well as its extensive versions, e.g., intersecting cortical model (ICM) [16] and spiking cortical model (SCM) [17], [18], has been successfully proposed and widely used to deal with the issue of image fusion. In essence, ICM and SCM are both the improved versions of the traditional PCNN model. The above three models are able to simulate the process of biological pulse motivation to capture the inherent information of source images. However, the models mentioned above all have a common limitation, e.g. inefficiency, due to the complex mechanism and a number of parameters requiring being setting.

In addition, the second category is based on the multi-scale analysis (MSA) theory. Discrete wavelet transform (DWT) has been regarded as an ideal fusion technique. However, further researches indicate that DWT still has its inherent limitations. First, it is merely good at capturing point-wise singularities, but the edge expression performance is poor. Second, it captures limited directional information only along vertical, horizontal and diagonal directions [19]. In order to overcome the drawbacks of DWT, a series of extensive improved models have been proposed, such as quaternion wavelet transform (QWT) [6], ridgelet transform (RT) [20], curvelet directional transform (CDT) [21], quaternion curvelet transform (QCT) [7], contourlet transform (CT) [22] and shearlet transform (ST) [23]. However, the performance of the above models is severely limited because of the absence of the shift-invariance property introduced by the down-sampling procedure. The shift-invariance extension of CT, namely non-subsampled contourlet transform (NSCT) [1], [24] has been explored and used, but its computational complexity is rather higher compared with aforementioned MSA techniques. Easley et al. proposed an improved version of ST called non-subsampled shearlet transform (NSST) [25] which not only has higher capability of capturing the feature information of the input images, but also costs much lower computational resources compared with NSCT. In spite of relatively good performance of preserving the details of the source images, MSA may produce brightness distortions since spatial consistency is not well considered in the fusion process [26].

In this paper, a novel technique with biochemical ion exchange model (BIEM) is proposed to conduct the fusion of multi-focus images. The core idea consists of three phases. Firstly, the BIEM theory is presented motivated by the nutrient uptake process of the root in biochemistry, and then the pixels from source images are viewed as the ions with different valences and polarities, respectively. Secondly, spatial frequency (SF) [27] is a reflection of the entire definition level of the image. Since the object of multi-focus image fusion is to replace the out-of-focus regions with the in-focus ones, SF can be chosen as the determinant of the valence. By completing the comparison of the valence value from left-top to right-bottom in the image, an iterative process is finished and a corresponding fused image can be obtained. Thirdly, the iteration will not stop until the growth momentum of the new structural similarity index measure (SSIM) namely the Piella’s metric [28] value of the subsequent fused image ends. Experiments are conducted on different multi-focus datasets to verify the effectiveness and superiorities of the propose technique over those of other current typical ones. The salient contributions of this paper can be summarized as follows.

  • BIEM, which stems from the perspective of the biochemical theory, is firstly proposed and used as the fusion framework for multi-focus images in this paper.

  • For obtaining the fused image, SF is used for deciding the valence value. The superiority of SF lies in that the main purpose on multi-focus image fusion is to substitute the clear pixels for blurred ones, and SF is a representative metric of the definition assessment ones.

  • SSIM has been well acknowledged as an objective metric of the fusion performance assessment, with which the iteration number can be adaptively determined without manual intervention.

  • Furthermore, the proposed technique has been also extended for the fusion of noisy source images.

Besides the contributions mentioned above, the work is also a valuable contribution to the area of soft computing. From a certain perspective, image fusion can be viewed as an application of the computing theory. The computing can be mainly categorized into two types including conventional computing (hard computing) and soft computing, which were both firstly proposed by Zadeh. As a traditional method of solving engineering problems, hard computing has several key features namely certainty and precision. However, hard computing is unsuitable for dealing with many problems because the real world is rich in uncertainty. Soft computing differs from hard computing in that, unlike hard computing, it is tolerant of imprecision, uncertainty, partial truth, and approximation. In effect, the role model for soft computing is the human mind. The guiding principle of soft computing is to exploit the tolerance for imprecision, uncertainty, partial truth, and approximation to achieve tractability, robustness and low solution cost. It can simulate the biochemical process of intelligent systems in nature to effectively handle routine tasks.

A large number of references [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44] involving both image processing and soft computing have been published in the past several years. The principal constituents of soft computing are fuzzy logic, neural network, probabilistic reasoning, genetic algorithm, chaos theory and so on. It is a partnership in which each of the partners contributes a distinct methodology for addressing problems in its domain.

Different from the conventional mechanism of liver biopsy, Kayaalti et al. [29] proposed a novel non-invasive, low-cost and relatively accurate method for determining the liver fibrosis stage. They analyzed the texture features of liver CT images with different methods including DWT, support vector machines (SVM) and so on. In order to solve the problem of multi-level image thresholding, a novel synergetic differential evolution (SDE) technique is proposed in reference [30]. In comparison with some current methods, SDE is much more effective and efficient. For sake of fusing the infrared and visible images, the dual-tree DWT model was constructed and used to complete the fusion process in reference [31]. Experimental results verify the effectiveness in terms of subjective and objective performances. In reference [32], double recurrent perceptual boundary detection neural (dPREEN) is proposed to extract the contours with perceptual significance and surface perception from color images. Compared with the typical models, the biologically inspired dPREEN model generated favorable and accurate contours. Reference [33] gave a new contourlet hidden Markov Tree and PCNN based fusion approach for remote sensing images. Kavitha et al. [34] proposed a simplified version of the traditional PCNN model called s-PCNN. s-PCNN utilized the hybrid soft computing architecture to complete the fusion process of medical images. Similarly, reference [35] improved the classic PCNN model with quantum-behaved particle swarm optimization to realize the successful fusion of multimodal medical images. Wang et al. [36] proposed a novel method for image segmentation based on hidden Markov tree and pyramidal dual-tree directional filter bank (PDTDFB). Compared with the current decomposition mechanisms, PDTDFB has not only efficient implementation and high angular resolution, but also low redundant ratio and shiftable subbands. Singh et al. [37] conducted a detailed review on image segmentation based on soft computing techniques. In reference [38], behavioral patterns were determined through knowledge-based modelling via soft computing technique. Experimental results indicate that the proposed technique does not need more features, but its performance is much better. The inspection of textile defects is always a tough issue. An interval type-2 fuzzy system was proposed for defects recognition in reference [39]. It gave us a good example based on the theory of soft computing. Mansouri et al. [40] combined some soft-computing methods together including genetic algorithm, SVM and fuzzy inference system to form a scheme which proves to be a systematic and potentially rapid damage detection technique for earthquake. Reference [41] provided a novel technique to video processing especially the de-interlacing task. Experimental results demonstrate that the proposed one has better performance and lower computational costs. A deep and detailed survey on the role of the different soft computing tools for the extraction of transcriptional genetic regulatory networks was conducted in reference [42]. The use of stereo vision techniques for 3D biological data gathering is affected by several restrictions. Otero et al. [43] proposed an effective soft computing based technique to improve the uncertainty volumes. In order to detect the invasive species in time, an automated system for image analysis- and soft computing-based detection was devised and proposed in reference [44]. Simulation results show that the proposed scheme has much higher detecting rate than the current typical ones.

Based on the analyses and discussions mentioned above, we can draw some general conclusions. Firstly, the field of soft computing is broad, and it covers so many aspects such as fuzzy logics, SVM, hidden Markov Tree, a series of biologically inspired models, and so on. Secondly, soft computing proves to be effective and feasible in the field of image processing including image fusion. As a result, we have reasons to believe that the soft computing based techniques have great potentials in dealing with the multi-focus image fusion.

In this paper, we propose the BIEM theory, which is motivated by the nutrient uptake process of the root in biochemistry and can be viewed as a member of soft computing. Experimental results demonstrate that, compared with several other soft computing methods such as fuzzy logics and a neural network, BIEM has obvious superiorities. Consequently, this work not only provides a novel solution to multi-focus image fusion, but also promotes the development of the system of soft computing theories.

The rest of this paper is organized as follows. The BIEM theory is presented in Section 2 followed by the fusion framework of the multi-focus images based on BIEM in Section 3. Experimental results with relevant analysis are reported in Section 4. Conclusions are summarized in Section 5.

Section snippets

Biochemical ion exchange model

This section provides the concept of the respiration in biochemistry, on which the BIEM theory is presented.

Fusion framework of multi-focus images based on BIEM

By comparing the fusion process and the cellular respiration, we can find that there are many similarities existing in terms of the principle. In this section, we have described the fusion framework of multi-focus images based on BIEM. The fusion course can be carried out via the framework. For simplicity, we take a pair of multi-focus source images for example, which are denoted by A and B respectively. It is noteworthy that A is regarded as the benchmark, and the source images should be

Experimental results with relevant analysis

In this section, several groups of experiments are conducted based on the proposed technique and some other current typical ones. By comparing these techniques from the perspectives of both subjective visual and objective numerical ways, the superiority of the proposed technique is shown convincingly.

This section consists of four parts. Section 4.1 describes the brief of the experimental setup, e.g., the techniques used to be compared with the proposed one, the basic information of the source

Conclusions

In this paper, a novel fusion framework is presented to deal with the issue of multi-focus images, which is based on biochemical ion exchange model. For fusion, the original source images are regarded as the inputs of the BIEM model, and each pixel is seen as the ion with different valence and polarity. The spatial frequency metric is used to calculate the valence for each ion. The whole fusion process can be adaptively regulated by the new structural similarity index measure. In our simulation

Acknowledgements

The authors would like to thank the anonymous reviewers and editors for their invaluable suggestions. The work was supported in part by the National Natural Science Foundations of China under Grant 61309008, 61309022, 61373116, 61472302 and 61473237, in part by the Natural Science Foundation of Shannxi Province of China under Grant 2014JQ8049, in part by the Foundation of Science and Technology on Information Assurance Laboratory under Grant KJ-15-102, and the Natural Science Foundations of the

References (49)

  • J. Saeedi et al.

    Infrared and visible image fusion using fuzzy logic and population-based optimization

    Appl. Soft Compt.

    (2012)
  • S.Y. Yang et al.

    Contourlet hidden Markov tree and clarity-saliency driven PCNN based remote sensing images fusion

    Appl. Soft Compt.

    (2012)
  • X.Z. Xu et al.

    Multimodal medical image fusion using PCNN optimized by the QPSO algorithm

    Appl. Soft Compt.

    (2016)
  • X.Y. Wang et al.

    Color image segmentation using PDTDFB domain hidden Markov tree model

    Appl. Soft Compt.

    (2015)
  • A. Verikas et al.

    Automated image analysis- and soft computing-based detection of the invasive dinoflagellate Prorocentrum minimum (Pavillard) Schiller

    Expert Syst. Appl.

    (2012)
  • J. Tian et al.

    Adaptive multi-focus image fusion using a wavelet based statistical sharpness measure

    Signal Process.

    (2012)
  • Y. Yang et al.

    Multimodal sensor medical image fusion based on type-2 fuzzy logic in NSCT domain

    IEEE Sens. J.

    (2016)
  • M. Ghahremani et al.

    Remote sensing image fusion using ripplet transform and compressed sensing

    IEEE Geosci. Remote Sens. Lett.

    (2015)
  • B. Yang et al.

    Multifocus image fusion and restoration with sparse representation

    IEEE Trans. Instrum. Meas.

    (2010)
  • Y. Yang et al.

    Multifocus image fusion based on NSCT and focused area detection

    IEEE Sens. J.

    (2015)
  • S. Pertuz et al.

    Genaration of all-in-focus images by noise-robust selective fusion of limited depth-of-field images

    IEEE Trans. Image Process.

    (2013)
  • L. Cao et al.

    Multi-focus image fusion based on spatial frequency in discrete cosine transform domain

    IEEE Signal Process. Lett.

    (2015)
  • Q. Zhang et al.

    Robust multi-focus image fusion using multi-task sparse representation and spatial context

    IEEE Trans. Image Process.

    (2016)
  • P.J. Burt et al.

    Enhanced image capture through fusion

    Proc Conf. Computer Vision

    (1993)
  • Cited by (9)

    • Multi-focus image fusion: A Survey of the state of the art

      2020, Information Fusion
      Citation Excerpt :

      For each aspect, a brief overview is given as follows. Conventional activity level measurements like variance, SF, EOG, EOL, SML are also frequently employed in pixel-based spatial domain methods [147–157]. In addition, there are also a number of pixel-based methods adopt the image decomposition approaches used in transform domain methods as the focus measure, such as QWT [158,159], NSCT [155], ND filtering [160], ICA [157], SR [90,92,93,161,162], robust principle component analysis (RPCA) [163], structure tensor [164,165], etc.

    • Fractal dimension based parameter adaptive dual channel PCNN for multi-focus image fusion

      2020, Optics and Lasers in Engineering
      Citation Excerpt :

      Furthermore, a block or region may have both focused and defocused pixels. Some well-known spatial domain based methods are image matting [20], random walk (RW) [21], depth information [18], biochemical ion exchange model [22], bilateral gradient [23], quad-tree [19], feature descriptor [15], artificial neural network (ANN) [24], pulse coupled neural network (PCNN) [25], convolutional neural network (CNN) [26], boundary finding [27], image partition [28], fuzzy logic [29], PCNN + RW [30], etc. In transform domain based methods, any transformation algorithm is first applied on source images to convert them from spatial domain to transform domain.

    • Technique for image fusion based on PCNN and convolutional neural network

      2018, Lecture Notes on Data Engineering and Communications Technologies
    View all citing articles on Scopus
    View full text