Bayesian inference for multiband image segmentation via model-based cluster trees
Introduction
Clustering and segmentation in an image analysis context have a long history [19]. Objectives include: quantization of data values for later use with a codebook in a compression context; targeting delivery to display devices supporting small, bounded pixel data value depth; as a preliminary to object and feature detection and analysis in images; and as a basis for other image processing operations such as image registration and archiving.
We will use the terms clustering or quantization to refer to determining clusters among image grayscale or pixel values. In the case of multiband images, the grayscale pixel values are multidimensional. This simply implies that in multiband data clustering we are dealing with clustering in multidimensional space, i.e. we are dealing with a form of vector quantization. Multiband images include the case of color images, with bands associated with red, green and blue colors, or a large number of alternative color formatting schemes. As opposed to clustering or quantization, the term segmentation is used when neighborhood or spatial influence information is incorporated into the modeling. Ideally, we could impose as a necessary objective that all segments be spatially contiguous. In practice, we take this as a sufficient objective. Multiband images are also referred to as multispectral or multichannel or hyperspectral images.
In this paper, we propose a new method for multiband image clustering, called model-based cluster trees. This combines maximum likelihood estimation of finite mixture models with Bayesian model selection. For segmentation, a Markov neighborhood dependency model is used to include adjacency or local influence. The model-based clustering tree algorithm operates recursively on the image bands. First it clusters or segments the pixels on the basis of the first band. Then, using the second selected band, it clusters each of the clusters found in the first stage. Bayesian model selection is used at each stage to determine the number of clusters or segments, so that the data are used to decide adaptively the extent to which the tree is pruned.
The resulting method allows the number of quantization levels or numbers of segments to be chosen on the basis of the data. If the number of quantization levels is predetermined (see, e.g. [23]), the method can easily handle this as a special case. Given that image bands are processed in sequence, it is helpful if the image bands have some inherent order. In chromaticity/luminosity color space, such an order can make use of the fact that chromaticities convey far less perceptual information than does the luminosity (see, e.g. [32]). Such an order can be readily accommodated in our approach. In more general cases, we impose an order on image bands which will be helpful for interpretation or further processing of the clustered or segmented output.
We can readily accommodate noise in our image data. This is implied by image features taken as realizations of distributional models. Explicit noise components are incorporated into our modeling as discussed in earlier work of ours [4]. Our MR software package [17] provides multiband image noise filtering, together with compression, functionality. See also chapter 6, ‘Multichannel data’, in [30].
We can accommodate a very small number of classes (clusters or segments) for the pixels, or a large number. A small number of classes may be needed as a preliminary to a data interpretation, or high-level vision stage of the analysis. A large number of classes may be needed when high fidelity to the original image is required.
A major motivation for a cluster tree results from use of model-based clustering in cases like multiband segmentation in Earth observation [22]. Notwithstanding the Occam razor parsimony principle of a small number of clusters, it may be found that a larger number of clusters does greater justice to the data. Then, however, it may be necessary to further analyze the clusters found. A cluster tree approach is an appropriate way to do this.
The simple tree structure given by a quadtree can be valuable, in particular for permitting Markov modeling both spatially and in scale [7]. However, two problems arise with such a simple tree structure: firstly, there is a sharp discontinuity at the boundaries between quadtree cells; and secondly the quadtree is quite a crude data-driven structure.
A further motivation for our cluster tree approach is that model-based Gaussian fitting of arbitrary multiband data is often unstable and algorithmically non-robust. The reason for this is singularity brought about by the following: (i) individual clusters or segments that are of small cardinality; (ii) correlation, possibly local, between bands; and (iii) relatively ‘flat’ background that is not covered by the detector, in particular in medical imaging. Some of these issues are discussed by us in [22].
In Section 2, we describe the model-based cluster trees methodology. In 3 Some algorithm properties, 4 Discussion of alternative approaches, we discuss aspects of algorithm design and properties. In Section 5, we will exemplify where the model-based tree approach is particularly important, and show how this algorithm performs exceedingly well in practice.
Section snippets
Model-based cluster trees
Our basic framework is that of model-based clustering, as described, for example, by Fraley and Raftery [11], [12]. In this methodology, a finite mixture of normal distributions is fit to the data by maximum likelihood estimation using the EM algorithm, the number of groups is chosen using Bayesian model selection, and if hard clustering is desired, each pixel is assigned to its most likely group a posteriori. Model-based cluster trees produces a clustering of multivariate data by clustering on
Band ordering
In this section, we follow closely Tate [31] who considers the band ordering problem for compression of multispectral images.
We consider the problem of clustering on one band coordinate, assuming that this band presents good clustering properties, followed by clustering on a second band based on the first band clustering, and so on. If c1i is the ith cluster from the first band, then we seek clusters c2j such that , i.e. Ji is a partition of cluster c1i. Similarly, we proceed to a
Discussion of alternative approaches
We will consider the characteristics of three different case studies. The third one provides the theme of Section 5 to follow.
The first case study [7] relates to six Hubble Space Telescope NICMOS (Near Infra-Red Camera and Multi-Object Spectrometer) infrared images (0.8–2.5 μm) of the M82 region. M82 will also be the focus of our third study below. M82 is the nearest starburst galaxy at a distance of 11 million light years from Earth. M82, cigar shaped, is bright (magnitude 6.9). In it, massive
Appraisal
We took the band in Fig. 3 as our point of departure. Segmentation of it was carried out for varying numbers of segments. This segmentation used a Markov model and ICM-based component mixture fit, as described in 2.4 Spatial segmentation and a modified Bayes information criterion, 2.5 An information criterion with spatial interaction, PLIC, 2.6 Model-based segmentation/clustering trees algorithm above. The PLIC criterion provides a basis for the selection of the best segmentation. This provides
Discussion
We have shown how a Bayesian modeling approach, model-based cluster trees, can lead to excellent results in the area of multiband image clustering. Formal underpinnings for such an algorithm facilitate choice of system parameters (e.g. number of clusters) which in a general setting would be set arbitrarily.
This approach allows us to carry out information fusion from multiband image data in a fully integrated way. We have discussed where and when this approach is particularly appropriate.
The
Acknowledgements
Raftery's research was supported by NIH Grant 1R01CA094212-01 and ONR Grant N00014-01-10745. Murtagh acknowledges helpful discussions within the IDHA and MDA projects supported by the French Ministry of Education's ‘Data Masses’ program, and also the European project iAstro, COST Action 283.
References (32)
- et al.
Linear flaw detection in woven textiles using model-based clustering
Pattern Recognition Letters
(1997) - et al.
Gaussian parsimonious clustering models
Pattern Recognition
(1995) - et al.
Multiband segmentation based on a hierarchical Markov model
Pattern Recognition
(2004) - et al.
Quantization from Bayes factors with application to multilevel thresholding
Pattern Recognition Letters
(2003) - et al.
Bayesian segmentation and clustering for determining cloud mask images
Statistical analysis of non-lattice data
Statistician
(1975)Statistical analysis of dirty pictures
Journal of the Royal Statistical Society, Series B
(1986)- et al.
Model-based methods for textile fault detection
International Journal of Imaging Science and Technology
(1999) - et al.
Detecting features in spatial point processes with clutter via model-based clustering
Journal of the American Statistical Association
(1998) - et al.
Maximum likelihood from incomplete data via the EM algorithm
Journal of the Royal Statistical Society, Series, B
(1977)
How many clusters? Which clustering method? Answers via model-based cluster analysis
The Computer Journal
Model-based clustering, discriminant analysis, and density estimation
Journal of the American Statistical Association
Model selection and the principle of minimum description length
Journal of the American Statistical Association
Bayes factors
Journal of the American Statistical Association
Mixture Models
Cited by (24)
Dirichlet Gaussian mixture model: Application to image segmentation
2011, Image and Vision ComputingCitation Excerpt :Accurately segmented images provide more and useful information for diagnosis and quantitative analysis. However, automated segmentation [1,2] is still a very challenging research topic, due to overlapping intensities and low contrast in images, as well as noise perturbation. In literature, different methodologies proposed for image segmentation include mean shift [3,4], clustering methods [5,6], graph based techniques [7–10], partial differential equations (PDE) based segmentation techniques [11,12], and region competition [13].
Bayesian mixture models of variable dimension for image segmentation
2009, Computer Methods and Programs in BiomedicineCitation Excerpt :In image segmentation applications, the Bayesian model selection criterion takes into account spatial neighborhood information, and is termed pseudo-likelihood information criterion (PLIC) [7]. However, in this kind of applications model-based Gaussian fitting has been found to be unstable and algorithmically non-robust [1]. Additional correction mechanisms of computational nature have been devised to optimize estimates, such as model-based cluster trees [1].
Application of circular laser vision sensor (CLVS) on welded seam tracking
2008, Journal of Materials Processing TechnologyHierarchical Matching and Regression with Application to Photometric Redshift Estimation
2016, Proceedings of the International Astronomical UnionHAL 9000: Skynet’s Risk Manager
2023, arXiv