Convergence of an EM-type algorithm for spatial clustering
Introduction
Spatial clustering aims to find classes composed of objects which are both similar according to some measure and geographically close. When classical clustering algorithms (e.g., the EM algorithm for Gaussian mixture estimation) are used for partitioning spatial data, the resulting classes will often be spatially very mixed.
In geology, sociology, image analysis, and in a wide range of other fields, spatial clustering techniques are widely used for finding homogeneous zones. Satellite images are often segmented in order to determine different zones of interest (e.g. forests, cities or rivers). In this particular case, the objects are pixels described by a gray scale or color intensity. Another example consists of statistics describing the number of sick persons per geographic unity (e.g. town or country) which may be used for delimiting different zones of risk.
Several methods exist for taking spatial information into account in a clustering process:
- •
Modifying existing clustering algorithms (Legendre, 1987; Lebart, 1978; Openshaw, 1977). This is done by specifying which objects are neighbors and allowing an object to be assigned to a class if and only if this class already contains a geographical neighbor. This approach has the drawback of producing classes which are necessarily geographically connected. This means that one class is bound to correspond to a single spatial region.
- •
Integrating the spatial information in the data set (Berry, 1966; Jain and Farrokhnia, 1991; Oliver and Webster, 1989). One example consists of considering the geographical coordinates as new variables describing the objects; another example is the filtering techniques that extract new features from the original variables which embody the spatial information.
- •
Choose a model which encompasses the spatial aspect of the data. Most of the time, this is equivalent to defining a criterion that includes spatial constraints. This approach comes mainly from image analysis where Markov random fields (Geman and Geman, 1984; Masson and Pieczinsky, 1993) are intensively used.
This paper aims to present a proof of the convergence of this algorithm for spatial clustering. Section 2introduces the Gaussian mixture model and describes the Neighborhood EM algorithm (NEM). Section 3is dedicated to the convergence proof. In Section 4, an illustrative example based on image segmentation is presented.
Section snippets
Gaussian mixture and clustering
The probabilistic approach to clustering is mainly based on Gaussian mixture models. In this framework (Celeux and Govaert, 1995), the objects to be classified are considered as a sample of independent random vectors. The vectors are drawn from a mixture of K Gaussian distributions:where the pk are the mixing proportions (for and ∑kpk=1) and denotes the density of a Gaussian distribution with parameter , being the mean
Estimation step
The method proposed in this section to perform the E-step is inspired from the Hathaway (1986) formulation of the EM algorithm and can be also related to the work of Neal and Hinton (1993). We suggest using the fixed point method to find the classification matrix which maximizes the criterion .
The necessary optimality Kuhn–Tucker conditions take the following form:where is the Lagrangian of that takes
An application to biological images
Let us illustrate the usefulness of the NEM algorithm1 with an application to image segmentation.
Let us consider the following biological experiment: a sample of living cells is laid on a nutritive substance. After a few days new living cells appear and form a thin but visible layer around the original sample. Biologists are interested in determining the surface of
Concluding remarks
The choice of the penalizing coefficient β remains the main difficulty in applying the NEM algorithm. In the preceding example we have used our experience to determine the “optimal” β coefficient. When such a procedure is not possible, it would be useful to have an automatic estimation of this parameter. This subject still needs further research.
A particularity of the NEM algorithm consists of providing a fuzzy partition of the data. This may be interesting in some applications where region of
References (19)
- et al.
Gaussian parsimonious clustering models
Pattern Recognition
(1995) An iterative Gibbsian technique for reconstruction of m-ary images
Pattern Recognition
(1989)Another interpretation of the EM algorithm for mixture distributions
J. Statist. Probab. Lett.
(1986)- et al.
Unsupervised texture segmentation using Gabor filters
Pattern Recognition
(1991) - Ambroise, C., 1996. Approche probabiliste en classification automatique et contraintes de voisinage. Ph.D. Thesis,...
- Ambroise, C., Dang, M., Govaert, G., 1996. Clustering of spatial data by the EM algorithm. In: Proceeding of...
- Berry, B.J.L., 1966. Essay on commodity flows and the spatial structure of the Indian economy. Research Paper 111,...
- et al.
Fuzzy random fields and unsupervised image segmentation
IEEE Trans. Geosci. Remote Sensing
(1993) - et al.
Maximum likelihood from incomplete data via the EM algorithm
J. Roy. Statist. Soc.
(1977)
Cited by (63)
Unsupervised quality control of segmentations based on a smoothness and intensity probabilistic model
2021, Medical Image AnalysisTowards justifying unsupervised stationary decisions for geostatistical modeling: Ensemble spatial and multivariate clustering with geomodeling specific clustering metrics
2018, Computers and GeosciencesCitation Excerpt :The goal is to generate classes that are spatially contiguous and have distinct multivariate properties. Two general strategies have been applied: 1) some form of neighborhood constraint to modify relatedness of distant and uncorrelated samples (Oliver and Webster, 1989; Ambroise and Govaert, 1998; Fouedjio,; Romary et al., 2015); or 2) generating a secondary dataset calculated from the original data with local autocorrelation statistics (Scrucca, 2005). Oliver and Webster (1989) justified the variogram model as a method to increase the relatedness of nearby points in clustering.
Extended likelihood approach to multiple testing with directional error control under a hidden Markov random field model
2016, Journal of Multivariate AnalysisSemi-supervised Latent Block Model with pairwise constraints
2022, Machine LearningModel-based Poisson co-clustering for Attributed Networks
2021, IEEE International Conference on Data Mining Workshops, ICDMW