CAPKM++2.0: An upgraded version of the collaborative annealing power $k$-means++ clustering algorithm

Abstract

The collaborative annealing power k-means++ (CAPKM++) clustering algorithm has been recently proposed based on multiple modules by minimizing annealed power-mean functions. This paper presents an upgraded version of CAPKM++ called CAPKM++2.0. Different from CAPKM++ where the anchor points of surrogate functions for majorizing the power-mean functions are re-initialized and minimized repeatedly after annealing, CAPKM++2.0 re-initializes the weights of the majorization function during annealing. In addition, unlike CAPKM++ that minimizes the majorization function of the power-mean sum, CAPKM++2.0 adds an inner loop to minimize the power-mean sum iteratively and locally at every annealing step. Ablation study results are discussed to justify the adoption of the power-mean and the collaboration of multiple modules. Experimental results on sixteen benchmark datasets are elaborated to demonstrate the superior clustering performance of the upgraded algorithm compared with its predecessor and six other mainstream algorithms in terms of cluster validity indices and algorithmic complexities.

Introduction

As an important procedure of data processing, clustering is to group similar data into homogeneous clusters [1], with widespread applications, such as image segmentation [2], [3], text mining [4], bioinformatic analysis [5]. Numerous clustering algorithms have been proposed and they may be classified into several categories from different perspectives, as shown in Fig. 1; e.g., divisive hierarchical clustering [6], agglomerative hierarchical clustering [7], $k$-means [8], [9], [10], $k$-medoids [11], $k$-harmonic means [12], spectral clustering [3], [13], [14], distribution clustering [15], [16], density clustering [17], [18], subspace clustering [19], [20], [21], feature-weighted clustering [22], [23], [24], probabilistic clustering [25], fuzzy clustering [26], [27], [28], cardinality-constrained clustering [29], [30], [31], capacitated clustering [32], [33], must-link and cannot-link constrained clustering [34], and rank-constrained clustering [35].

Lloyd’s algorithm [8] is a classic $k$-means algorithm that minimizes the sum of squared distances between each point and its assigned cluster center in a greedy way. The $k$-means and $k$-means-type algorithms are very popular and widely used in data clustering and analysis, owing to their efficiency and simplicity. However, their clustering qualities vary depending on the initialization of centers. To overcome this limitation, many alternative methods have been proposed, such as initialization improvements [11], [36], [37], [38] and objective function improvements [9], [12], [39], [40], [41]. Of particular interest, the power $k$-means (PKM) algorithm [9] clusters data by minimizing the majorization function of an annealed power-mean sum. Nevertheless, its clustering performance still depends on initialization. To eliminate the initialization dependence, CAPKM++ employs multiple PKM modules initialized by using $k$-mean++ and re-initialized repeatedly and collaboratively using a particle swarm optimization rule after each annealing process [10]. Although CAPKM++ outperforms many baselines, the clustering efficiency could be further improved by carrying out cluster re-initialization during the annealing process.

In this paper, we propose an upgraded version of CAPKM++ called CAPKM++2.0. Instead of re-initializing cluster centers after annealing, CAPKM++2.0 re-initializes the weights in the majorization function during annealing. Additionally, CAPKM++2.0 minimizes the power-mean functions directly rather than their majorization function as in PKM and CAPKM++. The novelties of this work are summarized as follows.

• Carrying out re-initialization during annealing rather than after it enables more efficient clustering with reduced spatial and temporal complexities.

• Minimizing the power-mean sum instead of its majorization function enables to further improve cluster performance in terms of algorithmic efficiency.

The remainder of this paper is organized as follows. Section 2 provides necessary preliminary information on problem formulation, PKM, and CAPKM++. Section 3 describes the proposed CAPKM++2.0 algorithm. Section 4 reports experimental results on sixteen datasets. Section 5 concludes the paper.