Uncertain fuzzy self-organization based clustering: interval type-2 fuzzy approach to adaptive resonance theory

Abstract

Conventional unsupervised learning algorithms require knowledge of the desired number of clusters beforehand. Even if such knowledge is not required in advance, empirical selection of the parameter values may limit the adaptive capability of the algorithm, thereby restricting the clustering performance. An inherent uncertainty in the number and size of clusters requires integration of fuzzy sets into a clustering algorithm. In this paper, we propose a type-1 (T1) fuzzy ART method that adaptively selects the vigilance parameter value, which is critical in determining the network dynamics. This results in improved clustering performance due to the added flexibility in dynamic selection of the number of clusters with the use of fuzzy sets. To further manage the uncertainty associated with memberships, we extend the proposed T1 fuzzy ART with adaptive vigilance to an interval type-2 (IT2) fuzzy ART method. Type reduction and defuzzification are then performed using the KM algorithm to obtain a defuzzified vigilance parameter value. We evaluate our proposed methods on several data sets to validate their effectiveness.

Introduction

In most unsupervised clustering algorithms, the desired number of clusters may not be estimated efficiently by the algorithm itself, since this value is required as an input parameter, such as in K-means clustering, or computation of complex statistics must be performed for this decision, such as the CH-index [2] and gap statistic [45] for hierarchical clustering. For this reason, various methods have been proposed to achieve self-organizing clustering, such as the adaptive resonance theory (ART) [4].

ART networks have emerged as intelligent and autonomous learning algorithms due to the following properties: self-organization, self-stabilization, plasticity, and real-time learning, where numerous network variants have been proposed, such as fuzzy ART [6], ARTMAP [5], ARTMAP-pi, dART, dARTMAP [3], and falcon ART [25], to name a few.

Although ART-based methods may provide a satisfactory baseline solution to the problem of assigning dynamic number and size of clusters, their performance is still restricted due to the requirement of selecting an empirical vigilance parameter value. For this reason, integrating fuzzy sets with ART methods, as in fuzzy ART [6], have proved to be successful in imitating the ambiguity in the number and size of clusters. They have since been utilized in a number of applications in diverse fields such as gene expression analysis [46] and image processing [8].

In order to improve the clustering performance of conventional ART, we propose a method to define the vigilance parameter as a function of the relative distance between the current centers of the candidate cluster and the input vector. Such a definition is based on the radial nature of the clusters present in most real datasets. The proposed vigilance function can then be viewed as a type-1 (T1) fuzzy membership function, where for each incoming pattern a suitable vigilance value may be obtained.

For any pattern recognition application, it may not always be possible to extract perfect knowledge from a pattern set. This ambiguity may lead to an uncertain choice of parameters values to represent data that may result in the formation of inappropriate prototypes. To mitigate this problem, many researchers have applied type-2 (T2) fuzzy methods to several applications, for example, medical diagnosis [18], community transport scheduling [19], reactive robot navigation [14], [26], chemical analysis [50], and survey processing [20], to name a few. Accordingly, the uncertainty associated with the vigilance parameter may be managed by extending it to a T2 fuzzy set (FS). In general, T2 FSs are capable of modelling various uncertainties that cannot be properly managed by T1 FSs, at the cost of added computational complexity.

To reduce this complexity, interval type-2 (IT2) FSs were proposed, since all secondary grades in these FSs are uniformly weighted (i.e., all equal to one). These MFs are also convenient for defuzzification, since the T1 fuzzy MF obtained after type reduction is an interval, which may be characterized by two fixed values. In light of this, we propose an IT2 fuzzy ART approach to further improve the performance of the proposed T1 fuzzy ART using the adaptive vigilance method, by incorporating the associated uncertainty of the data using upper and lower vigilance membership functions.

Our key contributions in this paper may be summarized as follows.

• We propose novel methods (T1 fuzzy ART with adaptive vigilance and IT2 fuzzy ART approach) for dynamic clustering that also takes into account the uncertainty associated with clusters in most real world applications.

• We model the vigilance parameter in ART clustering using T1 fuzzy and IT2 fuzzy MFs, and exploit the KM algorithm for defuzzification. Our method differs from conventional fuzzy ART in that the MF may be selected heuristically depending upon an estimate of the number and size of the clusters.

• We argue that the IT2 fuzzy approach may be suitable for extending any clustering algorithm that requires dynamic selection of clusters, due to its inherent ability to model uncertainties.

The remainder of this paper is organized as follows. We provide a brief review of the related research in Section 2. In Section 3, we discuss the conventional fuzzy ART algorithm and the effect of the vigilance parameter on the network dynamics. We also summarize the concepts of IT2 fuzzy sets, type reduction, and defuzzification. In Section 4, we describe our proposed methods, namely T1 fuzzy ART and IT2 fuzzy ART. Experimental results showing the effectiveness and improved performance of the algorithms are presented in Section 5. We conclude with a discussion of future work in Section 6.