$\alpha$-cut induced Fuzzy Deep Neural Network for change detection of SAR images

Highlights

• SWT based image fusion facilitates the identification of the trend of changes thanks to its shift invariance property.

• We put forward a novel $\alpha$-cut induced fuzzification layer in the neuro fuzzy architecture.

• $\alpha$-cut induced Fuzzification is employed to enhance the changed information and to reduce the effect of speckle noise.

• We reformulate FDNN in order to learn spatial representations under cognitive uncertainty.

Abstract

Change detection (CD) is a process of identifying dissimilarities from two or more co-registered multitemporal images. In this paper, we have introduced a $\alpha$-cut induced Fuzzy layer to the Deep Neural Network ($\alpha$FDNN). Deep neural networks for change detection normally rely on the pre-classified labels of the clustering. But the pre-classified labels are more coarse and ambiguous, which is not able to highlight the changed information accurately. This challenge can be addressed by encapsulating the local information and fuzzy logic into the deep neural network. This takes the advantage of enhancing the changed information and of reducing the effect of speckle noise. As the first step in change detection, a fused difference image is generated from the mean and log ratio image with the advent of Stationary Wavelet Transform (SWT). It not only eliminates the impact of speckle noise but also it has good ability to identify the trend of change thanks to the shift invariance property. Pseudo classification is performed as the next step using Fuzzy C Means (FCM) clustering. Then, we apply reformulated $\alpha$-cut induced Fuzzy Deep Neural Network to generate the final change map which facilitates a final representation of data more suitable for the process of classification and clustering. It also results into a noteworthy improvement in the change detection result. The efficacy of the algorithm is analyzed through the parameter study. Experimental results on three Synthetic Aperture Radar (SAR) datasets demonstrate the superior performance of the proposed method compared to state-of-the art change detection methods.

Introduction

Change Detection is explicated as the task of identifying dissimilarities of an entity by incessantly examining it [1]. Despite the numerous studies, this field of research still stumbles on its significance due to its massive appliance. It establishes its ample applications in various fields, such as remote sensing, disaster evaluation, estimation of land-use patterns, urban-growth monitoring, medical diagnosis, and video surveillance. Change detection can also provide essential information to help in policy making, area planning and efficient land organization.

Detecting changes in the earth surface involves monitoring, comparing and analyzing two multitemporal remote sensing images obtained from the same landscape at different time periods. The process involves capturing two images at different times or from different viewpoints. The first step involves preprocessing, which contains mainly procedures such as denoising, registration etc., Secondly the difference image is being generated. Difference Image (DI) indicates the difference between the two images based on the pixel variance. The third step analyzes the difference image by segmenting the difference image into two classes viz changed and unchanged.

The Earth observation satellite is used to acquire images with much soaring spatial and spectral resolutions. SAR data is the most widely used image modality in remote sensing change detection as it has the ability to penetrate through clouds and even in the absence of sun light.

Change detection methods are normally classified as either supervised or unsupervised according to the nature of data processing. The former one is based on a supervised classification method, which requires the availability of a ground truth in order to derive a suitable training set for the learning process of classifiers. The later approach performs change detection by making a direct comparison of two multitemporal images without incorporating any a priori​ information.

As SAR images are easily affected by speckle noise, removal of noise becomes a mandatory pre-processing step, which in turn helps to increase the change detection accuracy. Ratio operator is the popular technique to generate difference image in SAR images. Zhuang et al. [2], [3] had used Spatio-temporal adaptive neighborhood ratio and Improved neighborhood ratio operators to reduce the effect of speckle noise. Total variation denoising model was put forwarded by Lou et al. [4] in order to diminish the speckling effects of SAR images.

Ma et al. [5] applied wavelet-based fusion algorithm to generate the final change detection map. The fusion method can put away the background and highlight the changed regions. Due to joint information representation at the spatial–spectral domain, the Discrete wavelet transform became the most popular approximation in image fusion. Contourlet transform [6] was applied on SAR images by Zhu et al. and it was able to reduce the effect of speckle noise and thereby eliminate the difficulties in change detection in river course. Contourlet transform has advantages like shift invariance and directional selectivity when compared to wavelet transform. But the demerit is that it had a higher computational complexity. In this paper, we choose the SWT based image fusion because of its invariance property to translation and shift in change detection [7].

Analysis of difference image to produce change detection binary map is the last step in the process. Clustering and threshold methods are widely used for this in literature. Mishra et al. [8] added local similarity information into FCM. Gong et al. [9] applied an improved Fuzzy Local Information C Means(FLICM) algorithm which can accurately calculate the damping level of neighboring pixels. Tian et al. [10] modified the distance measure of the objective function, which considers the neighboring intensity and the texture information. Ghosh et al. [11] used FCM and Gautafson–Kessel clustering algorithm for CD in remote sensing images. Gong et al. [12] reformulated the FCM by introducing a new fuzzy factor into the objective function. Li et al. [13] used Multi-objective fuzzy clustering method for change detection. Genetic Particle Swarm Optimization based method was addressed by Chen et al. [14]. They used Ratio of Mean to Variance as the fitness function.

Liu et al. [15] have proposed artificial immune system which used the local information and fuzzy energy thereby made their model noise free. Ghosh et al. [16] introduced the modified self-organizing feature map (MSOFM) network to produce change detection map. Acclimatizing on the local contextual information of pixel’s neighborhood L. Ke et al. [17] derived the decision threshold. In contrast a significant test algorithm based on maximizing a posterior was raised by inflicting a weight to each pixel in the block to increase the change detection accuracy. Ma, W.et al. [18] applied multi scale image to train the model which employed gcForest by combining the gradient information with the probability map.

Semisupervised SAR image change detection was broached by An et al. [19]. In this work they used Discriminative Random Fields. Another semi supervised approach using combined sparse fusion and constrained K-means was attempted by Anisha et al. [20]. Zhenxuan Li et al. [21] put forwarded a new difference measure based on Gabor-Wavelet. The coefficient of variation in the Markov Random field neighborhood is able to discriminate changed pixel from unchanged pixel. Hybrid Conditional Random Field (HCRF), was proposed by Lv et al. [22] that combined the traditional random field with object based technique. It helped in a better way to depict the spectral–spatial information.

Nowadays, researchers turn their interest towards deep learning based approaches [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44] because of its ability to model the nonlinearity. Change detection using deep methods has two steps. Through deep neural network, initially two images for change detection can be transformed into a feature space, in which their features are more consistent. Next, the discrimination of the feature vector into binary map will take place. The effectiveness of feature representation directly impacts the change detection accuracy.

Gong et al. [23] suggested change detection in SAR images based on deep neural networks, which had integrated unsupervised feature learning and supervised fine-tuning to obtain the change detection. Deep learning algorithms produce good CD accuracy on heterogeneous images, i.e. images acquired from different sensors. Liu et al. [24] introduced a thresholding algorithm which is applied on heterogeneous images, through an unsupervised convolution coupling network. Gong et al. [25] converted heterogeneous images to homogeneous by building a coupled Generative Adversarial Networks (GAN) with the Variational Auto Encoder (VAE) thereby improved CD accuracy.

Zhang et al. [26] combined Deep Belief Network (DBN) with feature change analysis in order to better depict the changed area. Zhao et al. [27] attempted to check out the characteristic feature of the DI by applying DBN and revised Back Propagation algorithm. Lyu et al. [28] took the idea of using spectral information rather than spatial information to analyze the datum. Instead of exercising the deep neutral network, they merged two-dimensional Convolutional Neural Network (CNN) and one-dimensional Recurrent Neural Network (RNN). By integrating CNN with a RNN, Mou et al. [29] generated spectral–spatial feature representations along with analysis of temporal independence of the images.

Gong et al. [30] investigated the use of modeling techniques with Sparse Auto Encoder, CNN for unsupervised clustering. These methods were helpful in extracting key changes and to restrain the external noises. A Supervised Contractive Auto encoders (SCAEs) combined with FCM clustering is proposed in [31] to detect changes in SAR images. Authors of [32] introduced a Dual dense Convolutional Network (DCN), where two deep convolutional networks with dense connectivity in convolution layers were designed to detect changes between images. Salman et al. [33] had used the deep neural networks for change detection in the forest area. Gong et al. [34] conceived a coupling translation network for change detection of heterogeneous images.

Li et al. [35] applied a combined differential image and Residual U-net for urban building change detection. L. Xu et al. [36] demonstrated image object segmentation by semantically employing U-net network to segment the post-phase remote sensing image. S. Saha et al. [37] suggested that coherent multitemporal deep feature hypervectors attained from pretrained CNN used to obtain change vectors. B. Du et al. [38] supported a deep slow feature analysis model that utilized two symmetric deep networks for projecting the input data of bi-temporal imagery. In turn the slow feature analysis module stifles the unchanged components. P. Zhang et al. [39] introduced a Homogeneous Image (HI) Difference Representation Learning (DRL) network (HI-DRLnet) which discovers Gaussian-distributed and discriminative difference representations for nonchange and different types of changes.

An accelerated Genetic Algorithm strategy was implemented in parallel by Cai-Hong Mu et al. [40] to analyze and optimize undetermined pixels through search space decomposition. Li, H., Gong [41] reported a stacked denoising autoencoders model with self-paced learning.

Recently, in the literature, hybrid soft computing techniques are also evolved. Fuzzy logic has good knowledge to overcome the uncertainty of the ambiguous data; it is used in combination with deep neural networks [42]. Hierarchical fused fuzzy deep neural network has been presented by Deng et al. [43]. They have used the architecture for brain image segmentation and for classification. We have reformulated the Fuzzy Deep Neural Network (FDNN) by introducing $\alpha$-cut in the fuzzification part.

On the basis of the aforementioned analysis and to the best of our knowledge, in the literature, there is no common method available that takes advantages of both wavelet fusion and deep fuzzy approaches for change detection. The proposed methodology consists of three stages, namely, Difference image generation using SWT, Pre-classification using Fuzzy C Means clustering, Fuzzy $\alpha$-cut based feature fusion and generation of change map using Deep Neural Network (DNN) as follows.

• Difference image generation: Mean Ratio and log ratio are applied on the two multitemporal images which is then fed to SWT in order to obtain the fused difference image.

• Pre-classification using FCM: FCM is applied on the fused image which in turns produces change, no-change or undetermined classes.

• Feature fusion and Generation of change map: The main focus of this phase is to normalize the fused and pre-classified image using $\alpha$-cut induced Fuzzy Layer and as a result produce a feature map. Then, DNN is applied on the feature map in order to classify the changed and unchanged pixels.

The major contributions of our paper are as follows:

• We present a SWT based image fusion which facilitates the identification of the trend of changes thanks to its shift invariance property.

• We propose a novel $\alpha$-cut induced fuzzification layer in the neuro fuzzy architecture which integrates local information and fuzzy logic for addressing the changed information to ease the effect of speckle noise.

• We use DNN for generating the change map that learns spatial representations under cognitive uncertainty with the help of the back propagation learning algorithm.

The rest of this paper is structured as follows. In Section 2, the problem statements and the proposed algorithm is described in detail. Section 3 elucidates the test and trial results on multitemporal images to verify the feasibility of the method. Finally, the conclusion is drawn in Section 4.