On the Number of Clusters in Cluster Analysis

Abstract

Clustering is a fundamental tool for analyzing the structure of feature spaces. It has been applied to various fields such as pattern recognition, information retrieval and so on. Many studies have been done on this problem and various kinds of clustering methods have been proposed and compared (e.g., [1]). Some clustering algorithms partition the points in the feature space into the given number of clusters, and others generate hierarchical structure of clusters. Either type of clustering algorithms is not able to compare the partitions consisting of different number of clusters and users must specify the optimal number of clusters. However, when applying a clustering method, we don’t often know how many clusters exist in the given data. This paper discusses the framework for comparing partitions that consist of different number of clusters