A tree-based incremental overlapping clustering method using the three-way decision theory

doi:10.1016/j.knosys.2015.05.028

Knowledge-Based Systems

Volume 91, January 2016, Pages 189-203

https://doi.org/10.1016/j.knosys.2015.05.028 Get rights and content

Abstract

Existing clustering approaches are usually restricted to crisp clustering, where objects just belong to one cluster; meanwhile there are some applications where objects could belong to more than one cluster. In addition, existing clustering approaches usually analyze static datasets in which objects are kept unchanged after being processed; however many practical datasets are dynamically modified which means some previously learned patterns have to be updated accordingly. In this paper, we propose a new tree-based incremental overlapping clustering method using the three-way decision theory. The tree is constructed from representative points introduced by this paper, which can enhance the relevance of the search result. The overlapping cluster is represented by the three-way decision with interval sets, and the three-way decision strategies are designed to updating the clustering when the data increases. Furthermore, the proposed method can determine the number of clusters during the processing. The experimental results show that it can identifies clusters of arbitrary shapes and does not sacrifice the computing time, and more results of comparison experiments show that the performance of proposed method is better than the compared algorithms in most of cases.

Introduction

Most of existing clustering algorithms usually analyze static datasets in which objects are kept unchanged after being processed [1], [2]. However, in many practical applications, the datasets are dynamically modified which means some previously learned patterns have to be updated accordingly [3], [4]. Although these approaches have been successfully applied, there are some situations in which a richer model is needed for representing a cluster [5], [6]. For example, a researcher may collaborate with other researchers in different fields, therefore, if we cluster the researchers according to their interested areas, it could be expected that some researchers belong to more than one cluster. In these areas, overlapping clustering is useful and important as well as incremental clustering.

For this reason, the problem of incremental overlapping clustering is addressed in this paper. The main contribution of this work is an incremental overlapping clustering detection method, called TIOC-TWD (Tree-based Incremental Overlapping Clustering method using the Three-Way Decision theory). The proposed method introduces a new incremental clustering framework with three-way decision using interval sets and a new searching tree based on representative points, which together allows to obtain overlapping clusters when data increases. Besides, the TIOC-TWD introduces new three-way strategies to update efficiently the clustering after multiple objects increases. Furthermore, the proposed method can dynamically determine the number of clusters, and it does not need to define the number of cluster in advance. The above characteristics make the TIOC-TWD appropriate for handling overlapping clustering in applications where the data is increasing.

The experimental results show that the proposed method not only can identify clusters of arbitrary shapes, but also can merge small clusters into the big one when the data changes; the proposed method can detect new clusters which might be the result of splitting or new patterns. Besides, more experimental results show that the performance of proposed method is better than the compared algorithms in most of cases. We note that a short version of this work had been appeared in the RSCTC-2014 Workshop on the Three-Way Decisions and Probabilistic Rough Sets [7].

Section snippets

Related work

Nowadays, there are some achievements on the incremental clustering approaches. Ester et al. [8] put forward the IncDBSCAN clustering algorithm based on the DBSCAN. After that, Goyal et al. [9] proposed the derivation work which is more efficient than the IncDBSCAN because it is capable of adding points in bulk to existing set of clusters. Patra et al. [10] proposed an incremental clustering algorithm based on distance and leaders, but the algorithm needs to search the whole data space to find

Three-way decision clustering

To define our framework, let a universe be $U = {x_{1}, \dots, x_{n}, \dots, x_{N}}$ , and the resulting clustering scheme $C = {C_{1}, \dots, C_{k}, \dots, C_{K}}$ is a family of clusters of the universe. The x_n is an object which has D attributes, namely, $x_{n} = (x_{n}^{1}, \dots, x_{n}^{d}, \dots, x_{n}^{D})$ . The $x_{n}^{d}$ denotes the value of the d-th attribute of the object x_n, where $n \in {1, \dots, N}$ , and $d \in {1, \dots, D}$ .

We can look at the cluster analysis problem from a decision making perspective. For crisp clustering, it is a typical two-way decision; meanwhile for overlapping clustering

The TIOC-TWD clustering method

The processing of the proposed TIOC-TWD method is illustrated in Fig. 1. In fact, we also devise an overlapping clustering algorithm using three-way decision strategy for the initial static data, which is based on a graph of representative points by calculating the similarity between representative regions. It is called Algorithm 1 and described in Section 4.2.

Evaluation indices and datasets

We evaluate the proposed TIOC-TWD clustering approach through the following experiments. All the experiments are performed on a 2.67 GHz computer with 4 GB memory, and all algorithms are programmed in C++. The quality of the final clustering is evaluated by the traditional indices such as the accuracy, F-measure [42] and NMI [43], where the objects in boundary regions are deemed to be positive regions to fit these common formula.

Table 5 gives the summary information about the datasets and the

Conclusions

Existing clustering approaches are either restricted to crisp clustering or static datasets. In order to develop an approach to deal with overlapping clustering as well as incremental clustering, this paper proposed a new tree-based incremental overlapping clustering method using the three-way decision theory, called TIOC-TWD.

This paper first introduced three-way decision clustering to represent the overlapping clustering as well as crisp clustering, and described the problem of incremental

Acknowledgment

This work was supported in part by the National Natural Science Foundation of China under Grant Nos. 61379114 & 61272060.

References (44)

A.K. Jain
Data clustering: 50 years beyond k-means
Pattern Recogn. Lett.
(2010)
G. Peters et al.
Soft clustering-fuzzy and rough approaches and their extensions and derivatives
Int. J. Approx. Reason.
(2013)
H.L. Sun et al.
Incorder: incremental density-based community detection in dynamic networks
Knowl.-Based Syst.
(2014)
S. Lee et al.
Self-adaptive and dynamic clustering for online anomaly detection
Expert Syst. Appl.
(2011)
A.A. Abbasi et al.
A survey on clustering algorithms for wireless sensor networks
Comput. Commun.
(2007)
H.Z. Ning et al.
Incremental spectral clustering by efficiently updating the eigen-system
Pattern Recogn.
(2010)
R. Gil-García et al.
Dynamic hierarchical algorithms for document clustering
Pattern Recogn. Lett.
(2010)
A. Pérez-Suárez et al.
An algorithm based on density and compactness for dynamic overlapping clustering
Pattern Recogn.
(2013)
N. Labroche
Online fuzzy medoid based clustering algorithms
Neurocomputing
(2014)
D. Parmar et al.
Mmr: an algorithm for clustering categorical data using rough set theory
Data Knowl. Eng.
(2007)

G. Peters et al.

Dynamic rough clustering and its applications

Appl. Soft Comput.

(2012)

Y.Y. Yao

The superiority of three-way decisions in probabilistic rough set models

Inform. Sci.

(2011)

C. Luo et al.

Fast algorithms for computing rough approximations in set-valued decision system while updating criteria values

Inform. Sci.

(2015)

C. Luo et al.

Dynamic maintenance of approximations in set-valued ordered decision systems under the attribute generalization

Inform. Sci.

(2014)

N. Azam et al.

Analyzing uncertainties of probabilistic rough set regions with game-theoretic rough sets

Int. J. Approx. Reason.

(2014)

D.C. Liang et al.

Systematic studies on three-way decisions with interval-valued decision-theoretic rough sets

Inform. Sci.

(2014)

H. Yu et al.

An automatic method to determine the number of clusters using decision-theoretic rough set

Int. J. Approx. Reason.

(2014)

M. Chen et al.

Interval set clustering

Expert Syst. Appl.

(2011)

A. Pérez-Suárez et al.

A new overlapping clustering algorithm based on graph theory

H. Yu et al.

An incremental clustering approach based on three-way decisions

M. Ester, H.-P. Kriegel, J. Sander, M. Wimmer, X.W. Xu, Incremental clustering for mining in a data warehousing...

N. Goyal, P. Goyal, K. Venkatramaiah, P.C. Deepak, P.S. SANNOP, An efficient density based incremental clustering...

Cited by (202)

Weighted probability kernel multi-granularity three-way decision integrating GRA and its application in medical diagnosis
2024, Information Sciences
Three-way decision, an outstanding method to handle decision-making uncertainties, relies on essentially the loss functions derived from the Bayesian risk decision process. Actually, there are plentiful loss functions that depend on the subjective judgment of decision-makers under different decision scenarios, lacking uniform and objective measurement frameworks. This study pays attention to the real clinical diagnosis, and constructs a weighted probability kernel multi-granularity three-way decision method (WKMG-TWD) integrating gray relation analysis (GRA) over a multi-source heterogeneous decision information system (MHDIS). The method establishes a standardized data-driven calculation framework of loss functions. Foremost, the multi-kernel probabilistic similarity is defined and granularity's weights with knowledge consistency are explored. Subsequently, a weighted probability kernel multi-granularity rough set (WKMGRS) is constructed in this paper. Secondly, to introduce the three-way decision, this study proposes the cost-sensitive individual loss functions considering the correlation determined by GRA between decision objects and different decision classes. Ultimately, this study establishes and applies a three-way iterative classification model to hypertension diagnosis. The experimental results confirm the effectiveness and superiority of the model. The main contribution of this paper is twofold. One is to offer a uniform calculation framework for loss functions and granularity's weights. The other is to furnish invaluable guidance for solving complex medical decision-making problems.
Multi-level personalized k-anonymity privacy-preserving model based on sequential three-way decisions
2024, Expert Systems with Applications
K-anonymity is a widely used privacy-preserving technique which defends against linking attacks by suppression and generalization. The existing k-anonymity algorithms prevent attackers from illegally obtaining private information by constraining at least k records in an equivalence group. However, this unified anonymity method ignores individual differences and leads to a large amount of information loss. To this end, we introduce sequential three-way decisions into k-anonymity, using a dynamic k-value sequence instead of the fixed k-value to achieve personalized k-anonymity. Specifically, we first construct a hierarchical decision table for k-anonymity by attribute generalization trees and sensitive decision values provided with users. Then, we propose a multi-level personalized k-anonymity privacy-preserving model based on sequential three-way decisions, where we anonymize the partitioning granular data with a dynamic k-value sequence, respectively. Furthermore, we present three practical algorithms to implement the proposed model and discuss the differences between them. Finally, the experimental results demonstrate that the proposed model not only provides a more flexible anonymization method to achieve personalized anonymity, but greatly reduces the information loss. This study provides a complete framework for multi-level privacy protection and enriches the application of sequential three-way decisions.
Key grids based batch-incremental CLIQUE clustering algorithm considering cluster structure changes
2024, Information Sciences
In the network environment, data from various industries is dynamic and large-scale. Traditional clustering algorithms struggle to effectively utilize existing clustering results when faced with continuously evolving data, which makes the incremental grid-based clustering highly regarded. However, the existing incremental grid-based clustering algorithms fail to adequately consider the impact of newly added data on the original cluster structure. To address this issue, the key grids based batch-incremental CLIQUE clustering algorithm is proposed. The algorithm designates the incremental data mapping grids, which are or their neighbour girds are mixed with original data, as key grids to fully consider the cluster structure changes caused by the incremental data. Moreover, the cluster similarity coefficient based on grid features is introduced to measure density differences between the incremental data and the original clusters, and the cluster membership degree is defined to further consider the cluster membership of boundary sparse grid data and the identification of noise points. All of which ensures that the algorithm can adaptively create, merge or split clusters with the arrival of new data. Experimental results show that the proposed algorithm can adaptively adjust the cluster structure during incremental clustering, outperforming in accuracy and efficiency when clustering large-scale, dynamically changing data.
A dynamic three-way conflict analysis model with adaptive thresholds
2024, Information Sciences
Three-way conflict analysis model, based on information systems, is about trisecting, getting coalitions and decision-making. Since the model is proposed by Yao, many models have been proposed to enrich the field of three-way conflict analysis. In this paper, we propose a new three-way conflict analysis model. Our work can be divided into four parts. First, we define new fuzzy granules and an new approximation operator to obtain better trisections. Second, we introduce dynamic information systems which are generalizations of classical information systems. The new information systems enable us to induce trisections dynamically. Third, a new Bayesian algorithm is proposed for thresholds-computing which plays an important role in inducing trisections. In the classical Bayesian approach, there are six original parameters given subjectively. And in our algorithm, the parameters are replaced by three centers and a distance measure. Finally, our new model is applied to tourism planning in two cases, one can be presented by a dynamic information system and the other one can be presented by a dispersed information system generalized from dynamic information system.
A two-stage adaptive consensus reaching model by virtue of three-way clustering for large-scale group decision making
2023, Information Sciences
As digitalization advances and societal patterns evolve, an increasing number of experts are becoming integral to the decision-making process. The realm of large-scale group decision making (LSGDM) has gained prominence in management sciences, offering effective solutions for real-world decision challenges. Given that experts from diverse fields possess varying degrees of expertise within LSGDM, their viewpoints exhibit considerable disparity. To attain decision outcomes that enjoy broad consensus, it becomes imperative to ensure equity and rationality within consensus models. However, a significant number of existing consensus models predominantly emphasize inter-subgroup consensus levels, inadvertently neglecting the significance of intra-subgroup consensus degrees. The incorporation of intra-subgroup consensus mechanisms holds pivotal importance for rational subgroup leader selection and opinion refinement among subgroups. Remarkably, this aspect has been underexplored in many prior studies. Addressing this gap, we introduce a novel approach called three-way clustering (TWC), inspired by the traditional k-means clustering technique, denoted as the TWC-KM method. By closely intertwining the clustering process with consensus mechanisms, we introduce a TWC-based two-stage adaptive consensus reaching (ACR) model that judiciously accounts for both intra-subgroup and inter-subgroup consensus aspects, aptly named the TWC-ACR model. Furthermore, we introduce an objective method for setting consensus thresholds, which considers the subgroup silhouette's size. Ultimately, the outcomes of our case study and numerical analysis compellingly demonstrate the superior efficacy and reliability of our devised model.
Tri-granularity attribute reduction of three-way concept lattices
2023, Knowledge-Based Systems
A real-world dataset often generates a huge concept lattice that may be difficult to understand and impractical to use. The principle of granular computing is that we can understand and use a complex whole at multiple levels of granularity. In light of this, we propose a unified tri-granularity model of various types of concept lattices, which allows us to examine a huge concept lattice at three levels of granularity (i.e., the elementary granularity, local granularity and global granularity). Attribute reduction plays a fundamental role in three-way concept analysis, it simplifies the expression of three-way concepts and thus contributes to a better perception of the knowledge in three-way concept lattices. The tri-granularity model suggests an opportunity to investigate the tri-granularity attribute reduction of three-way concept lattices. The existing research on attribute reduction works at the global granularity but pays little attention to the local granularity, or, even less, to the elementary granularity. Driven by these issues, we supply definitions and methods of local granularity and elementary granularity attribute reduction of three-way concept lattices. These newly proposed two levels of attribute reduction with the existing global granularity attribute reduction together provide a framework for the tri-granularity attribute reduction of three-way concept lattices. We further analyze the relationships among the three levels of attribute reduction. Moreover, the efficacy of our suggested approach is illustrated by an example via the trisections induced by three-way concepts. Finally, we design two tri-granularity attribute reduction algorithms whose effectiveness is further examined by numerical experiments.

View all citing articles on Scopus

View full text

A tree-based incremental overlapping clustering method using the three-way decision theory

Abstract

Introduction

Section snippets

Related work

Three-way decision clustering

The TIOC-TWD clustering method

Evaluation indices and datasets

Conclusions

Acknowledgment

Pattern Recogn. Lett.

Int. J. Approx. Reason.

Knowl.-Based Syst.

Expert Syst. Appl.

Comput. Commun.

Pattern Recogn.

Pattern Recogn. Lett.

Pattern Recogn.

Neurocomputing

Data Knowl. Eng.

Appl. Soft Comput.

Inform. Sci.

Inform. Sci.

Inform. Sci.

Int. J. Approx. Reason.

Inform. Sci.

Int. J. Approx. Reason.

Expert Syst. Appl.

A new overlapping clustering algorithm based on graph theory

An incremental clustering approach based on three-way decisions