Learning probabilistic models by conceptual pyramidal clustering

Abstract

Symbolic objects (Diday (1987, 1992), Brito, Diday (1990), Brito (1991)) allow to model data on the form of descriptions by intension, thus generalizing the usual tabular model of data analysis. This modelisation allows to take into account variability within a set. The formalism of symbolic objects has some notions in common with VL1, proposed by Michalski (1980); however VL1 is mainly based on prepositional and predicate calculus, while the formalism of symbolic objects allows for an explicit interpretation within its framework, by considering the duality intension-extension. That is, given a set of observations, we consider the couple (symbolic object — extension in the given set). This results from the wish to keep a statistics point of view. The need to represent non-deterministic knowledge, that is, data for which the values for the different variables are assigned a weight, led to considering an extension of assertion objects to probabilist objects (Diday 1992). In this case, data are represented by probability distributions on the variables observation sets. The notions previously defined for assertion objects are the generalized to this new kind of symbolic objects. Other extensions can be found in Diday (1992).

In order to obtain a clustering structure on the data, we propose a pyramidal symbolic clustering method. Methods of conceptual clustering have known a great improvement in the last years. These methods aim at obtaining a system of interpreted clusters. In the eighties a large number of conceptual clustering methods have been developed, mention should be made to Michalski (1980), Michalski, Diday and Stepp (1982), Fisher (1987). The pyramidal model (Diday (1984,1986), Bertrand (1986)) generalizes hierarchical clustering by presenting an overlapping instead of a partition at each level; pyramidal clustering produces a structure which is nevertheless much simpler than the lattice structure, since each cluster should be an interval of a total order θ (as a consequence, pyramids present no crossing in the graphic representation). So, the pyramidal model supplies a structure which is richer than hierarchical trees (in that it keeps more information), and that, compared to lattices, are easily interpretable.

The method we present builds a pyramid by an agglomerative algorithm. Each cluster is represented by a symbolic object whose extension — that is the set of elements verifying it — is the cluster itself. Hence a cluster is represented by a couple extension — intension.

An application on textual data modelled by probabilist objects (texts described by frequency of key-words) allowed to obtain a pyramid whose clusters (representing though groups of texts) are themselves represented by probabilist objects, we thus identify groups of texts refering to close subjects.