Elsevier

Applied Soft Computing

Volume 87, February 2020, 105928
Applied Soft Computing

Fuzzy C-Means clustering through SSIM and patch for image segmentation

https://doi.org/10.1016/j.asoc.2019.105928Get rights and content

Highlights

  • New similarity measure is put forward based on proposed DSSIM metric.

  • New coefficient is from new similarity measure and weighted sum distance of patch.

  • New algorithm is proposed based on the idea of image patch and new coefficient.

  • New algorithm performs better than related FCM algorithms for image segmentation.

Abstract

In this study, we propose a new robust Fuzzy C-Means (FCM) algorithm for image segmentation called the patch-based fuzzy local similarity c-means (PFLSCM). First of all, the weighted sum distance of image patch is employed to determine the distance of the image pixel and the cluster center, where the comprehensive image features are considered instead of a simple level of brightness (gray value). Second, the structural similarity (SSIM) index takes into account similar degrees of luminance, contrast, and structure of image. The DSSIM (distance for structural similarity) metric is developed on a basis of SSIM in order to characterize the distance between two pixels in the whole image. Next a new similarity measure is proposed. Furthermore, a new fuzzy coefficient is proposed via the new similarity measure together with the weighted sum distance of image patch, and then the PFLSCM algorithm is put forward based on the idea of image patch and this coefficient. Through a collection of experimental studies using synthetic and publicly available images, we demonstrate that the proposed PFLSCM algorithm achieves improved segmentation performance in comparison with the results produced by some related FCM-based algorithms.

Introduction

Image segmentation is embodied as a key task in many fields such as computer vision, pattern recognition, affective computing, and multimedia [1], [2], [3], [4], [5]. For example, Hernández et al. [6] extended the idea of residue properties, which helped to generate the image quantization table with regard to an arithmetic approach. Kushwaha and Welekar [7] investigated feature selection for content-based image retrieval, in which optimal features were obtained from the feature selection process realized by means of the genetic algorithm. Rezaie and Habiboghli [8] proposed a strategy for the detection of malignant and benign tumors on the CT scan images, where fractal segmentation was used. Image segmentation aims to divide image pixels into several non-overlapping regions, where the pixels in a given region exhibit similar characteristics while pixels positioned in different regions are different. Fuzzy sets [9], [10], [11], especially Fuzzy C-Means (FCM) clustering algorithms [12], [13], have been extensively employed to carry out image segmentation leading to the improved performance of the segmentation process. The “standard” FCM algorithm works well for most noise-free images, however it is sensitive to noise, outliers and other imaging artifacts. The main reasons behind these drawbacks lie in neglecting spatial context information.

Since the introduction of the FCM algorithm, it has attracted growing interest in the area of image segmentation. Tolias and Panas [14] presented a hierarchical fuzzy clustering-based image segmentation algorithm that was able to cope with nonstationarity and high correlations between pixels. Its performance was better than the possibilistic c-means (PCM) algorithm. Pham and Prince [15] introduced a multiplier field to propose a fuzzy segmentation algorithm for images that were subject to multiplicative intensity inhomogeneities. Wang et al. [16] incorporated the information-theoretic framework and adaptive spatial weighting factors into the FCM-type algorithms to enhance its robustness for image segmentation. Zhou et al. [17] presented a modified mode of the FCM algorithm for image segmentation, in which one used a simple way to update the cluster centers and partitioned the pixels by adding a new bias term into the FCM method. Ji et al. [18] proposed a novel fuzzy clustering approach for brain MR image segmentation. It employed the negative log-posterior regarded as the dissimilarity function, and introduced a new factor with the spatial direction, and finally incorporated the bias field estimation model into the optimized objective function. Adhikari et al. [19] presented a conditional spatial FCM algorithm for MRI image segmentation. It was constructed by the introduction of conditioning effects imposed by an auxiliary variable related to each pixel as well as spatial information into the membership functions. Chatzis and Varvarigou [20] combined the benefits of the hidden Markov random field (HMRF) with FCM, and established the HMRF-FCM algorithm for image segmentation, which utilized the spatial coherency expressing abilities of HMRF to enhance the FCM segmentation effect. Following the HMRF-FCM algorithm, Liu et al. [21] emphasized the treatment of local information, and introduced region-level information to adjust the range and strength of interactive image pixels. This work was mainly aimed at segmentation of natural color images, and synthetic aperture radar images. In some studies, regularization terms were considered to control the effect of the membership functions. Li et al. [22] employed regularization with the entropy for the membership function. In [23], Miyamoto and Umayahara regularized the FCM function with a quadratic term. But, similar to the classical FCM algorithm, they are only related to the image intensity. Hou et al. [24] regarded a moving-average filter as the regularizer, so it adjusted the main function with the window average of neighborhoods. In [25], the FCM algorithm was improved by a regularizing functional via total variation (TV) related to gradient sparsity, and a regularization parameter was utilized to balance clustering and smoothing. This algorithm was found to be effective and robust in testing images affected by noise and missing data.

As an important improvement of FCM, the spatial and gray-level information were introduced into the generic FCM algorithm. Pham [26] employed a spatial penalty based on cross-validation for the FCM objective functions. The corresponding iterative algorithm was only slightly different from the FCM algorithm and allowed the estimation of spatially smooth membership functions. Ahmed et al. [27] proposed FCM_S, in which the basic formula of the FCM was adjusted to compensate for the intensity inhomogeneity and to determine the pixel labeling according to its immediate neighborhood. Furthermore, Chen and Zhang [28] put forward the FCM_S1 and FCM_S2 algorithms as simplified versions of FCM_S, which led to acceptable segmentation results. To speed up the image segmentation process, Szilagyi et al. [29] presented the enhanced FCM (EnFCM) algorithm. In this algorithm, a new image was generated from linearly weighted sum of the original image, and then the gray level histogram of the new image was used for further fuzzy clustering. Similar to EnFCM, Cai et al. [30] proposed a fast generalized FCM (FGFCM) clustering algorithm. This algorithm employed a local similarity measure according to local spatial closeness as well as intensity information, which formed a non-linearly weighted sum image and thus also was characterized by high computational speed. Following it, by virtue of simple local similarity measures, the FGFCM_S1 and FGFCM_S2 algorithms were also presented. Zhao et al. [31] presented a FCM algorithm with non-local spatial information obtained from a large image domain to form the spatial constraint term, in which the non-local spatial information of a pixel was achieved by employing the pixels with a similar configuration of the given pixel. Ma et al. [32] proposed an improved FGFCM algorithm with non-local spatial information, where local and non-local similarity measures were employed with an adaptive weight to balance their impact.

In the previous FCM algorithms, hyper parameters were usually required to control the balance of eliminating noise and retain image details. The values of these hyper parameters were selected experimentally through a trial-and-error method. To solve such problem, Krinidis and Chatzis [33] proposed the fuzzy local information c-means (FLICM) algorithm. FLICM is a special method with a sound and convincing idea, which incorporates the local spatial information and gray level information in a coherent manner. For the first time, FLICM puts forward a fuzzy coefficient Gki which is regarded as a fuzzy local (both spatial and gray level) similarity measure in order to guarantee noise robustness as well as retention of details. Moreover, FLICM is free from empirically adjustable parameters whose tuning usually creates a certain challenge. Based on these observations, FLICM is effective and efficient in the sense that it exhibits robustness in case of noisy images. However, it exhibits some disadvantages:

  • In the fuzzy clustering algorithm, one important issue is how to characterize the relationship between the image pixel xi and the cluster center vk. It is embodied as the key problem to deliver ideal segmentation result. Actually, FLICM only utilizes the distance d(xi,vk) to capture this dependency, which is the same as in the FCM algorithm. Strictly speaking, d(xi,vk) cannot adequately characterize such relationship, which only involves two values xi,vk without considering the overall characteristics of a more comprehensive character.

  • Another important factor is how to characterize the relationship between two pixels xi and xj. In the FLICM method, one uses only the spatial Euclidean distance dij between two pixels i and j to reflect this relationship. By looking more carefully at the essence of the problem, we should establish a general similarity measure between i and j vis-à-vis the entire image. Here dij is not sufficient to grasp the generalized characteristics of the segmented image.

Following the FLICM algorithm, some further improvements were proposed. Li et al. [34] presented the FCM algorithm with edge and local information (FELICM), which reduced the edge degradation by incorporating the weights of pixels within local neighbor windows. Gong et al. [35] put forward an improvement of FLICM algorithm (RFLICM), which employed the local coefficient of variation to replace the spatial distance as a local similarity measure. Then Gong et al. [36] proposed FCM clustering with local information and kernel metric (KWFLICM) algorithm by setting up a tradeoff weight fuzzy coefficient and a kernel metric, in which the fuzzy coefficient was simultaneously determined in the space distance of all neighboring pixels and their gray-level difference. Verma et al. [37] presented an improved intuitionistic FCM (IIFCM), which was concerned with the local spatial information under the intuitionistic fuzzy environment. Ji et al. [38] proposed FCM clustering with weighted image patch (WIPFCM), which considered image patches to replace pixels with a weighting scheme. Table 1 shows the main idea and highlights its merits along with shortcoming of the related fuzzy clustering algorithms such as FCM, FCM_S, FCM_S1, FCM_S2, EnFCM, FGFCM, FGFCM_S1, FGFCM_S2, FLICM, KWFLICM, IIFCM and WIPFCM.

Although there are some algorithms [34], [35], [36], [37], [38] enhancing the performance of the FLICM algorithm to some extent. However, they all employ the computing mechanism similar to FLICM, and it should be pointed out that these two disadvantages still exist. To alleviate them, we elaborate on further enhancements for this issue. In this paper, we will investigate the overall characteristics to characterize the relationship between xi and vk, and discover the general similarity measure between i and j, and develop a new image segmentation algorithm.

In order to solve such key problem, in this study, we put forward a new FCM algorithm, referred to as the patch-based fuzzy local similarity c-means (PFLSCM) algorithm. First, since the image patches incorporate more general information than image pixels, we use image patch to analyze the relationship between the image pixel and the cluster center, and then employ the weighted sum distance of image patch to measure the distance of the image pixel and the cluster center. Second, we propose a new local distance measure derived from the structural similarity (SSIM) index to compute the distance between two image pixels in the overall image, and then put forward a novel similarity measure. The new one conveys not only the spatial relationship of two image pixels but also the relationship related to luminance and contrast as well as structure of two patches revolved around them. Third, the PFLSCM algorithm is designed based on the idea of image patch, the novel similarity measure as well as the corresponding fuzzy coefficient. Lastly, we carry on experiments using synthetic, real-world and medical images with several types of noises, and it is found that the PFLSCM algorithm has better performance than other seven algorithms in terms of evaluation indicators and visualization effects.

Section snippets

Proposed method

First of all, we explore how to adequately utilize the characteristics of image pixels. From the general viewpoint, using the image patch can reveal more structure of image than individual pixels. As a result, the basic distance d(xi,vk) can be restructured into a weighted sum of image patch: r=1pωrd(xir,vkr).Here xir is the value of the point in an image patch (e.g., a window) located around xi, and p stands for the number of points in the image patch, while vkr is the new cluster centers (i=1

Experimental studies

All experiments include a comprehensive comparative analysis where we engage a number of clustering algorithms studied in the literature. Here we concentrate on the strategy, which incorporates the spatial and gray-level information together in the FCM algorithms. Therefore we compare the PFLSCM algorithm with several representative algorithms including EnFCM, FGFCM, FGFCM_S1, FGFCM_S2, and FLICM. Moreover, in several improvements of the FLICM, the KWFLICM algorithm becomes a successful

Conclusions

In the FCM clustering algorithm used for image segmentation, a crucial issue is how to appropriately characterize the relationship between the image pixel and the cluster center, as well as the relationship between two image pixels. To properly characterize these two relationships, a novel FCM algorithm called PFLSCM algorithm is put forward and investigated.

The research of the proposed PFLSCM approach is summarized as follows. To begin with, the idea of image patches is introduced for not only

Declaration of Competing Interest

No author associated with this paper has disclosed any potential or pertinent conflicts which may be perceived to have impending conflict with this work. For full disclosure statements refer to https://doi.org/10.1016/j.asoc.2019.105928.

Acknowledgment

This work was supported by the National Natural Science Foundation of China (Nos. 61673156, 61432004, U1613217, 61672202), the Fundamental Research Funds for the Central Universities of China (No. ACAIM190101), the Natural Science Foundation of Anhui Province (Nos.1408085MKL15, 1508085QF129), and the China Postdoctoral Science Foundation (Nos. 2012M521218, 2014T70585).

Yiming Tang received the Ph.D. degree from Hefei University of Technology in 2011. He is now an Associate Professor in Hefei University of Technology, and his research interests include clustering, image processing, fuzzy system, fuzzy logic, and affective computing.

He has published more than 50 papers in many journals and conferences. He serves an Associate Editor of JOURNAL OF MATHEMATICS AND INFORMATICS, a member of the Editorial Board of Mathematics and Computer Science, and also a senior

References (49)

  • JiZ.X. et al.

    Fuzzy c-means clustering with weighted image patch for image segmentation

    Appl. Soft Comput.

    (2012)
  • ChangH.H. et al.

    Performance measure characterization for evaluating neuroimage sSegmentation algorithms

    NeuroImage

    (2009)
  • MasulliF. et al.

    A fuzzy clustering based segmentation system as support to diagnosis in medical imaging

    Artif. Intell. Med.

    (1999)
  • ManinisK.K. et al.

    Convolutional oriented boundaries: From image segmentation to high-level tasks

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2018)
  • ChenC. et al.

    Partial membership latent Dirichlet allocation for soft image segmentation

    IEEE Trans. Image Process.

    (2017)
  • C. Rupprecht, L. Peter, N. Navab, Image segmentation in twenty questions, in: Proc. IEEE Conf. Comput. Vis. Pattern...
  • HernándezF.L. et al.

    Residue properties for the arithmetical estimation of the image quantization table

    Appl. Soft Comput.

    (2018)
  • KushwahaP. et al.

    Feature selection for image retrieval based on genetic algorithm

    Int. J. Interact. Multimed. Artif. Intell.

    (2017)
  • RezaieA.A. et al.

    Detection of lung nodules on medical images by the use of fractal segmentation

    Int. J. Interact. Multimed. Artif. Intell.

    (2017)
  • JavedU. et al.

    SAR image segmentation based on active contours with fuzzy logic

    IEEE Trans. Aerosp. Electron. Syst.

    (2016)
  • PedryczW. et al.

    An optimization of allocation of information granularity in the interpretation of data structures: Toward granular fuzzy clustering

    IEEE Trans. Syst. Man Cybern. B

    (2012)
  • TangY.M. et al.

    Variable differently implicational inference for R- and S-implications

    Int. J. Inf. Tech. Decis.

    (2016)
  • IzakianH. et al.

    Clustering spatiotemporal data: An augmented fuzzy c-means

    IEEE Trans. Fuzzy Syst.

    (2013)
  • Balla-ArabeS. et al.

    A fast and robust level set method for image segmentation using fuzzy clustering and lattice Boltzmann method

    IEEE Trans. Cybern.

    (2013)
  • Cited by (98)

    • Multi-stage generative adversarial networks for generating pavement crack images

      2024, Engineering Applications of Artificial Intelligence
    • Research on the standardization strategy of granular computing

      2023, International Journal of Cognitive Computing in Engineering
    View all citing articles on Scopus

    Yiming Tang received the Ph.D. degree from Hefei University of Technology in 2011. He is now an Associate Professor in Hefei University of Technology, and his research interests include clustering, image processing, fuzzy system, fuzzy logic, and affective computing.

    He has published more than 50 papers in many journals and conferences. He serves an Associate Editor of JOURNAL OF MATHEMATICS AND INFORMATICS, a member of the Editorial Board of Mathematics and Computer Science, and also a senior member of CAAI (Chinese Association for Artificial Intelligence) and a member of IEEE, CCF (China Computer Federation). He serves as a Professional Committee of Multiple-Valued and Fuzzy Logic of CCF, a Professional Committee of Rough Sets and Soft Computing of CAAI, a Professional Committee of Machine Learning of CAAI. He is a reviewer of several journals (including IEEE TRANSACTIONS ON FUZZY SYSTEMS, INFORMATION SCIENCES, ISPRS JOURNAL OF PHOTOGRAMMETRY AND REMOTE SENSING, SIGNAL PROCESSING). He is now a visiting professor at University of Alberta, Canada.

    Fuji Ren received the Ph.D. Degree in 1991 from Faculty of Engineering, Hokkaido University, Japan. He is a Professor and the Director of the Faculty of Engineering, the University of Tokushima. His research interests include artificial intelligence, fuzzy system, image processing, natural language processing, and affective computing.

    He serves as the Chang Jiang Scholar Endowed Chair Professor of Ministry of Education, the winner of Funds for Overseas Distinguished Young Scientists, the overseas evaluation expert of Chinese Academy of Sciences, and Haizhi experts of China Association for Science and Technology. Moreover, he is the conference founder and Chairman of International Conference on Natural Language Processing and Knowledge Engineering. He is also a member of the CAAI, IEEJ, IPSJ, JSAI, AAMT and a senior member of IEEE, IEICE.

    He is the Editor-in-Chief of INTERNATIONAL JOURNAL OF ADVANCED INTELLIGENCE, and a member of the Editorial Board of INTERNATIONAL JOURNAL OF INFORMATION TECHNOLOGY AND DECISION MAKING, INTERNATIONAL JOURNAL OF INFORMATION ACQUISITION. He is a Fellow of the Japan Federation of Engineering Societies.

    Witold Pedrycz received the M.Sc., Ph.D., and D.Sc. degrees from Silesian University of Technology, Gliwice, Poland, in 1977, 1980, and 1984, respectively. He is a Professor and the Canada Research Chair of computational intelligence with the Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB, Canada. He is also with the Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland, where he was elected as a foreign member in 2009. He also holds an appointment as Special Professor with the School of Computer Science, University of Nottingham, Nottingham, U.K. His current research interests include computational intelligence, image processing, fuzzy modeling and granular computing, knowledge discovery and data mining, fuzzy control, pattern recognition, knowledge-based neural networks, relational computing, and software engineering. He has authored or coauthored numerous papers. He is also the author of 14 research monographs that cover various aspects of computational intelligence and software engineering.

    He is the Editor-in-Chief of INFORMATION SCIENCES. He is also an Associate Editor of the IEEE TRANSACTIONS ON FUZZY SYSTEMS, IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS: SYSTEMS and APPLIED SOFT COMPUTING. He has edited a number of volumes, including Handbook of Granular Computing (Wiley, 2008). Prof. Pedrycz has been a member of numerous program committees of IEEE conferences in the area of fuzzy sets and neurocomputing. He received the prestigious Norbert Wiener Award from the IEEE Systems, Man, and Cybernetics Council in 2007. He received the IEEE Canada Computer Engineering Medal in 2008 and the Cajastur Prize for Soft Computing in 2009 from the European Centre for Soft Computing for pioneering and multifaceted contributions to granular computing.

    View full text