Interpretable interval type-2 fuzzy predicates for data clustering: A new automatic generation method based on self-organizing maps

Highlights

• A new clustering based on interval type-2 fuzzy predicates and SOMs is proposed.

• SOMs are automatically configured and trained.

• Fuzzy predicates are generated using cluster prototypes extracted from SOMs.

• Linguistic knowledge is obtained from the predicates automatically generated.

• The proposed method overcome existing clustering methods based on fuzzy predicates.

Abstract

In previous works, we proposed two methods for data clustering based on automatically discovered fuzzy predicates which were referred to as SOM-based Fuzzy Predicate Clustering (SFPC) [Meschino et al., Neurocomputing, 147, 47–59 (2015)] and Type-2 Data-based Fuzzy Predicate Clustering (T2-DFPC) [Comas et al., Expert Syst. Appl., 68, 136–150 (2017)]. In such methods, fuzzy predicates allow both data clustering and knowledge discovering about the obtained clusters. This last feature constitutes novelty comparing to other existing approaches and it is a major contribution in the data clustering field. Based on these previous methods, in the present paper a new automatic clustering method based on fuzzy predicates is proposed which uses Self-Organizing Maps (SOMs) and is called Type-2 SOM-based Fuzzy Predicate Clustering (T2-SFPC). The new method does not require any prior knowledge about the clustering addressed. First, a random partition is defined on the dataset to be clustered and SOMs are configured and trained using the resulting data subsets. Second, an automatic clustering approach is applied on the SOM codebooks, discovering representative data of the different clusters, which are called cluster prototypes. Third, interval type-2 membership function formed by Gaussian-shape sub-functions and fuzzy predicates are defined, allowing data clustering and its interpretation. The proposed method preserves all the advantages of the previous methods SFPC and T2-DFPC in relation to the knowledge extraction capabilities and their potential application on distributed clustering and parallel computing, but results obtained on several public datasets tested showed more compactness and separation of the clusters defined by the T2-SFPC, outperforming both the previous methods and the several classical clustering approaches tested, considering internal and external validation indices. Additionally, both clustering interpretation and optimization capabilities are improved by the proposed method when compared to the methods SFPC and T2-DFPC.

Introduction

Data clustering refers to grouping data according to a similar criterion [1] revealing hidden structures in data. It has multiple applications in very different fields such as: data mining, marketing, machine learning, bioinformatics, image segmentation, pattern recognition, among others; and new methods are continually proposed [2], [3], [4], [5]. Clustering methods are primarily designed to assign clusters to data, usually not requiring prior information about the expected results except for the number of clusters to be obtained, which is typically required. As a result, the outcome of the most common methods typically consists of a vector containing the corresponding cluster for each datum, including the cluster centroids or prototypes of the type of data corresponding to each of the clusters (called cluster prototypes).

Despite traditional approaches of data clustering are only used to group data; other potential applications can emerge. In fact, clustering can be addressed as a set of data analysis techniques which discover groups of similar data and their results can be exploited extracting information about them. Such information could be related to what the common properties of the data inside a same cluster are and how these properties differ from a cluster to another [3].

In this regard, Fuzzy Logic (FL), conceived as a natural extension of Boolean logic which introduces degrees of truth between 0 (completely false) and 1 (completely true), is able to model linguistic expressions and concepts, including imprecision and vagueness, being excellent for modeling and implementing human reasoning expressed by linguistic expressions, achieving interpretable clustering.

Typical FL models are based on fuzzy inference systems using IF-THEN rules considering approaches such as Mamdani and Takagi–Sugeno–Kang [6], [7], [8], [9], [10], [11], [12] applied in image classification, image segmentation, speech recognition, control, among others. Although widely used, a fuzzy inference system requires defining fuzzification, aggregation and defuzzification operators and its outcome is a continuous variable. Considering data clustering applications, these characteristics of the fuzzy inference systems become difficult to understand the relation between the data and their properties and the system outcome, i.e. the assigned cluster [2], [3]. On the other hand, models based on fuzzy predicates extend the Boolean predicates, modeling degrees of truth of predicates with values between 0 and 1 [13]. When applied on data clustering, fuzzy predicate models allow to implement knowledge about the clustering, explaining which values of each feature are related to each of the clusters and modeling these relationships using membership functions and predicates. Such models have been successfully applied in data clustering [2], [3], [14], [15], having the following features:

• Each cluster is explained by a fuzzy predicate interpreted as “The datum belongs to the cluster k”, being k a cluster; explaining which values (which characteristics) of each feature are related to each of the clusters.

• In order to assign clusters to data, degrees of truth of the predicates are computed for each datum using the membership functions and the fuzzy operators and in each case the cluster corresponding to the predicate with the maximum degree of truth is assigned to the datum.

• Resulting degrees of truth quantify in what grade each datum meets the characteristics required to belong to a cluster (i.e. how a datum is represented by their prototypes).

In the traditional approach, it is required knowledge of experts to define both the membership functions and the predicates. Once designed, the fuzzy predicates apply that knowledge for assigning clusters to data. Nevertheless, in recent works an approach alternative to the traditional one has been studied for data clustering which is based on the automatic generation of the membership functions and the fuzzy predicates by analyzing the data to be clustered. Such approach has an enormous and immediate advantage in relation with the traditional approach: it not only allows the data clustering, but also provides knowledge about the clustering obtained by interpreting the membership functions and the predicates generated. As a consequence, relevant information about data can be obtained, even when no prior information about the problem addressed is available [3].

In this regard, in the previous works [2], [3] we proposed two methods for data clustering through fuzzy predicates in which membership functions and fuzzy predicates are automatically generated from the data to be clustered. In [2], a method based on Self-Organizing Maps (SOMs) (a set of wide known unsupervised and nonparametric neural networks with remarkable abilities for dealing with noise, outliers, and missing values) is proposed called SOM-based Fuzzy Predicate Clustering (SFPC). In the SFPC, a SOM is automatically trained and set and, then, Fuzzy C-Means (FCM) [16] clustering is applied to the codebook of the SOM, extracting cluster prototypes. From these prototypes, membership functions and fuzzy predicates are defined, linguistically explaining the clusters. The method includes a variant where several SOMs are generated from data subsets and predicates obtained from the different SOMs are combined, which could be applied to distributed clustering. Predicates are used to perform the data clustering and some analysis of the interpretation of the membership functions and predicates is given.

In [3], the method called Type-2 Data-based Fuzzy Predicate Clustering (T2-DFPC) is introduced. Unlike the SFPC, interval type-2 FL is used which defines a degree of truth by an interval in [0, 1] called interval of truth values, instead of a number between 0 and 1 as in the case of type-1 FL, which adds additional degrees of freedom considering data clustering. Interval type-2 FL provides more appropriate models than type-1 FL for dealing with vagueness and imprecision about the data characteristics and can reduce the effect on cluster assignments in data affected by noise [3], [13], [17]. In the T2-DFPC, the cluster prototypes are extracted directly from data without using SOMs, combining FCM with the Bayesian Information Criterion (BIC) [18], [19], defining automatically the proper number of clusters in each case. Before the cluster prototype extraction, a random partition is performed on the data, obtaining disjoint subsets. The method is also suitable for distributed clustering. The T2-DFPC includes an analysis of the obtained membership functions and predicates, describing how the knowledge can be extracted. Additionally, it is also proposed a measure of intervals of truth values defining a methodology for interval comparing which allows the cluster assignment when interval type-2 fuzzy predicates are used.

Based on these two previous methods, in the present paper a new clustering method called Type-2 SOM-based Fuzzy Predicate Clustering (T2-SFPC) is proposed, which automatically generates interval type-2 membership functions and fuzzy predicates, allowing data clustering and knowledge discovery. The method proposed uses SOMs in order to obtain cluster prototypes, exploiting their advantages for noise, outliers, and missing values dealing, as in the SFPC; following the methodology used in that method for the automatic configuration and training of the SOMs. However, unlike the previous SFPC, in the method T2-SFPC, M SOMs are automatically configured and trained from M disjoint subsets defined by a random partition on the data, where $M\in \mathbb{N}$ is a method parameter. Once the M SOMs are defined, the clustering approach combining FCM with the BIC is applied not requiring knowing the number of clusters to be obtained. Once cluster prototypes are extracted, interval type-2 membership functions and fuzzy predicates are generated in a different way to that proposed in the T2-DFPC [3]. Specifically, the new proposal includes parametrizable interval type-2 membership functions, i.e. it is possible the optimization of the parameters of the membership functions provided that a specific goal is defined, for instance adopting a clustering quality measure. As a result of the proposed method, one fuzzy predicate is defined for each cluster. The clustering assignment is performed using the methodology introduced in [3] by means of the measure of interval of truth values.

The contribution of the proposed method is a new general methodology for data clustering, which can be applied to most of the clustering problems. The interval type-2 membership functions merge all knowledge extracted of the cluster prototypes from the M SOMs. The method T2-SFPC preserves all the characteristics of the previous methods, mainly those related to the SOM abilities for discovering natural data groupings and, also, to the knowledge discovery capabilities studied in the T2-DFPC. As in the T2-DFPC, linguistic expressions extracted from the predicates can be adapted to match the terminology of the domain experts, not requiring any prior knowledge about the dataset or the clustering problem addressed. Tests performed considering widely different datasets reveal better results from the proposed T2-SFPC than those obtained both from the SFPC, the T2-DFPC and classical clustering methods, meaning that the proposed method is an excellent clustering method choice when both data clustering and knowledge extraction from the clustering results are needed.

The rest of this paper is structured as follow. In Section 2, it is presented an analysis of the main existing papers concerning to SOMs used for data analysis as well as FL applications. Important concepts related to SOMs and interval type-2 FL are presented in Section 3 and, after that, the method proposed called T2-SFPC is explained in detail. In Section 4, experiments performed to the assessment of the proposed method are described and their results are presented, including an example of the interpretation of the obtained clustering in the case of segmentation of brain magnetic resonance images. Finally, in Sections 5 and 6, discussion and conclusions are presented, commenting on the results and the limitations of the method proposed as well as future work.