A New Implementation of the co-VAT Algorithm for Visual Assessment of Clusters in Rectangular Relational Data

Abstract

This paper presents a new implementation of the co-VAT algorithm. We assume we have an m×n matrix D, where the elements of D are pair-wise dissimilarities between m row objects O _r and n column objects O _c . The union of these disjoint sets are (N = m + n) objects O. Clustering tendency assessment is the process by which a data set is analyzed to determine the number(s) of clusters present. In 2007, the co-Visual Assessment of Tendency (co-VAT) algorithm was proposed for rectangular data such as these. co-VAT is a visual approach that addresses four clustering tendency questions: i) How many clusters are in the row objects O _r ? ii) How many clusters are in the column objects O _c ? iii) How many clusters are in the union of the row and column objects O _r  ∪ O _c ? And, iv) How many (co)-clusters are there that contain at least one of each type? co-VAT first imputes pair-wise dissimilarity values among the row objects, the square relational matrix D _r , and the column objects, the square relational matrix D _c , and then builds a larger square dissimilarity matrix D _{r ∪  c} . The clustering questions can then be addressed by using the VAT algorithm on D _r , D _c , and D _{r ∪  c} ; D is reordered by shuffling the reordering indices of D _{r ∪  c} . Subsequently, the co-VAT image of D may show tendency for co-clusters (problem iv). We first discuss a different way to construct this image, and then we also extend a path-based distance transform, which is used in the iVAT algorithm, to co-VAT. The new algorithm, co-iVAT, shows dramatic improvement in the ability of co-VAT to show cluster tendency in rectangular dissimilarity data.