Investigating diversity of clustering methods: An empirical comparison

Abstract

The paper aims to shed some light on the question why clustering algorithms, despite being quantitative and hence supposedly objective in nature, yield different and varied results. To do that, we took 10 common clustering algorithms and tested them over four known datasets, used in the literature as baselines with agreed upon clusters. One additional method, Binary-Positive, developed by our team, was added to the analysis. The results affirm the unpredictable nature of the clustering process, point to different assumptions taken by different methods. One conclusion of the study is to carefully choose the appropriate clustering method for any given application.

Introduction

The data mining field is advancing rapidly both in academia and in industry, and is at the technological forefront of data resource usage and management in organizations. Data mining deals with identifying interesting and useful patterns and associations within existing data, to reach new insights, which will add to the organization’s knowledge base. A variety of models and algorithms from different fields are being used, originating from statistics, artificial intelligence, neural networks, and databases, all coming under a general term – knowledge discovery and data mining (KDD).

Among the many techniques and models utilized in data mining are clustering and classification – two principal techniques which are the focus of empirical studies in this field. The goal of clustering is to organize the entities or objects into groups, in a manner, which results in minimal distances within each group, and maximum distances between the various groups [6]. Cluster analysis is thus the division of a collection of records into groups. This analysis is based on similarity. Describing this intuitively, we could say that the similarity between records within a valid cluster is greater than the similarity between these records and those belonging to a different cluster. The variety of techniques for representing data, measuring proximity (similarity) between records, and grouping records (algorithms) have produced a rich and sometimes confusing assortment of methods. A clear distinction is thus needed between classification and clustering before we dwell on the latter.

Classification is a supervised process – we are provided with a collection of pre-classified (labeled) records; the problem is to label the records that are as yet unlabelled. The labeled or “training” records are used to learn the attributes of a group, which in turn are used to label the new record. Clustering, on the other hand, is an unsupervised process. The problem is to group a given collection of unlabelled records into meaningful clusters. While the categories in classification are set externally, categories in clustering are determined by the clustering process itself, i.e., the various categories are derived from the dataset itself.

Clustering is useful in several exploratory pattern-analysis, grouping, decision-making and machine learning situations. However, in many such problems, there is little prior information (e.g. statistic models) available about the data where the decision maker has to make as few assumptions about the data as possible [13].

Despite being one of the central techniques in data mining, and its obvious advantages, clustering use is limited to academic research and data analysis, and as yet to gain wide use in commerce. There are several reasons for the clustering technique being accepted so slowly. Firstly, there is a standardization problem. Different clustering algorithms produce different clusters and there is no clear-cut and standard method to compare them. Secondly, the interpretation for the various clusters formed, and their implementation in the original environment is not defined. Managers and business users hardly know the value of what they can attain by performing clustering using whatever technique. Indeed, the clustering process is unpredictable and sometimes even inconsistent. Different programs generally divide the same dataset differently. This diversity makes the clustering process difficult. Furthermore, there is no clear way to measure and evaluate the quality of a clustering algorithm [7].

A large number of clustering algorithms are available and reported in the literature. This can easily confound a user trying to choose an algorithm, which suits his problem. Clustering techniques, while purely quantitative in nature, turn out to be quite unpredictable in their approach as well as outcome. Different clustering techniques hardly produce similar clusters from the same data, a puzzling phenomenon. This is possibly due to clustering algorithms driven by underlying assumptions, cluster formats, or different representation. And most of all, the outcome depends on definition of similarity measures and clustering criteria being used in the process. This paper aims to bring some light on such processes that produce unpredictable result by taking a range of algorithms (11 of them) and applying them to the same datasets (4 such sets). In all we ran 44 clustering experiments. Such experiments may yield some conclusions on the clustering process itself.