Bayesian inference for multiband image segmentation via model-based cluster trees

https://doi.org/10.1016/j.imavis.2005.02.002Get rights and content

Abstract

We consider the problem of multiband image clustering and segmentation. We propose a new methodology for doing this, called model-based cluster trees. This is grounded in model-based clustering, which bases inference on finite mixture models estimated by maximum likelihood using the EM algorithm, and automatically chooses the number of clusters by Bayesian model selection, approximated using BIC, the Bayesian Information Criterion. For segmentation, model-based clustering is based on a Markov spatial dependence model. In the Markov model case, the Bayesian model selection criterion takes account of spatial neighborhood information, and is termed PLIC, the Pseudolikelihood Information Criterion. We build a cluster tree by first segmenting an image band, then using the second band to cluster each of the level 1 clusters, and continuing if required for further bands. The tree is pruned automatically as a part of the algorithm by using Bayesian model selection to choose the number of clusters at each stage. An efficient algorithm for implementing the methodology is proposed. An example is used to evaluate this new approach, and the advantages and disadvantages of alternative approaches to multiband segmentation and clustering are discussed.

Introduction

Clustering and segmentation in an image analysis context have a long history [19]. Objectives include: quantization of data values for later use with a codebook in a compression context; targeting delivery to display devices supporting small, bounded pixel data value depth; as a preliminary to object and feature detection and analysis in images; and as a basis for other image processing operations such as image registration and archiving.

We will use the terms clustering or quantization to refer to determining clusters among image grayscale or pixel values. In the case of multiband images, the grayscale pixel values are multidimensional. This simply implies that in multiband data clustering we are dealing with clustering in multidimensional space, i.e. we are dealing with a form of vector quantization. Multiband images include the case of color images, with bands associated with red, green and blue colors, or a large number of alternative color formatting schemes. As opposed to clustering or quantization, the term segmentation is used when neighborhood or spatial influence information is incorporated into the modeling. Ideally, we could impose as a necessary objective that all segments be spatially contiguous. In practice, we take this as a sufficient objective. Multiband images are also referred to as multispectral or multichannel or hyperspectral images.

In this paper, we propose a new method for multiband image clustering, called model-based cluster trees. This combines maximum likelihood estimation of finite mixture models with Bayesian model selection. For segmentation, a Markov neighborhood dependency model is used to include adjacency or local influence. The model-based clustering tree algorithm operates recursively on the image bands. First it clusters or segments the pixels on the basis of the first band. Then, using the second selected band, it clusters each of the clusters found in the first stage. Bayesian model selection is used at each stage to determine the number of clusters or segments, so that the data are used to decide adaptively the extent to which the tree is pruned.

The resulting method allows the number of quantization levels or numbers of segments to be chosen on the basis of the data. If the number of quantization levels is predetermined (see, e.g. [23]), the method can easily handle this as a special case. Given that image bands are processed in sequence, it is helpful if the image bands have some inherent order. In chromaticity/luminosity color space, such an order can make use of the fact that chromaticities convey far less perceptual information than does the luminosity (see, e.g. [32]). Such an order can be readily accommodated in our approach. In more general cases, we impose an order on image bands which will be helpful for interpretation or further processing of the clustered or segmented output.

We can readily accommodate noise in our image data. This is implied by image features taken as realizations of distributional models. Explicit noise components are incorporated into our modeling as discussed in earlier work of ours [4]. Our MR software package [17] provides multiband image noise filtering, together with compression, functionality. See also chapter 6, ‘Multichannel data’, in [30].

We can accommodate a very small number of classes (clusters or segments) for the pixels, or a large number. A small number of classes may be needed as a preliminary to a data interpretation, or high-level vision stage of the analysis. A large number of classes may be needed when high fidelity to the original image is required.

A major motivation for a cluster tree results from use of model-based clustering in cases like multiband segmentation in Earth observation [22]. Notwithstanding the Occam razor parsimony principle of a small number of clusters, it may be found that a larger number of clusters does greater justice to the data. Then, however, it may be necessary to further analyze the clusters found. A cluster tree approach is an appropriate way to do this.

The simple tree structure given by a quadtree can be valuable, in particular for permitting Markov modeling both spatially and in scale [7]. However, two problems arise with such a simple tree structure: firstly, there is a sharp discontinuity at the boundaries between quadtree cells; and secondly the quadtree is quite a crude data-driven structure.

A further motivation for our cluster tree approach is that model-based Gaussian fitting of arbitrary multiband data is often unstable and algorithmically non-robust. The reason for this is singularity brought about by the following: (i) individual clusters or segments that are of small cardinality; (ii) correlation, possibly local, between bands; and (iii) relatively ‘flat’ background that is not covered by the detector, in particular in medical imaging. Some of these issues are discussed by us in [22].

In Section 2, we describe the model-based cluster trees methodology. In 3 Some algorithm properties, 4 Discussion of alternative approaches, we discuss aspects of algorithm design and properties. In Section 5, we will exemplify where the model-based tree approach is particularly important, and show how this algorithm performs exceedingly well in practice.

Section snippets

Model-based cluster trees

Our basic framework is that of model-based clustering, as described, for example, by Fraley and Raftery [11], [12]. In this methodology, a finite mixture of normal distributions is fit to the data by maximum likelihood estimation using the EM algorithm, the number of groups is chosen using Bayesian model selection, and if hard clustering is desired, each pixel is assigned to its most likely group a posteriori. Model-based cluster trees produces a clustering of multivariate data by clustering on

Band ordering

In this section, we follow closely Tate [31] who considers the band ordering problem for compression of multispectral images.

We consider the problem of clustering on one band coordinate, assuming that this band presents good clustering properties, followed by clustering on a second band based on the first band clustering, and so on. If c1i is the ith cluster from the first band, then we seek clusters c2j such that c1i=jJic2j, i.e. Ji is a partition of cluster c1i. Similarly, we proceed to a

Discussion of alternative approaches

We will consider the characteristics of three different case studies. The third one provides the theme of Section 5 to follow.

The first case study [7] relates to six Hubble Space Telescope NICMOS (Near Infra-Red Camera and Multi-Object Spectrometer) infrared images (0.8–2.5 μm) of the M82 region. M82 will also be the focus of our third study below. M82 is the nearest starburst galaxy at a distance of 11 million light years from Earth. M82, cigar shaped, is bright (magnitude 6.9). In it, massive

Appraisal

We took the band in Fig. 3 as our point of departure. Segmentation of it was carried out for varying numbers of segments. This segmentation used a Markov model and ICM-based component mixture fit, as described in 2.4 Spatial segmentation and a modified Bayes information criterion, 2.5 An information criterion with spatial interaction, PLIC, 2.6 Model-based segmentation/clustering trees algorithm above. The PLIC criterion provides a basis for the selection of the best segmentation. This provides

Discussion

We have shown how a Bayesian modeling approach, model-based cluster trees, can lead to excellent results in the area of multiband image clustering. Formal underpinnings for such an algorithm facilitate choice of system parameters (e.g. number of clusters) which in a general setting would be set arbitrarily.

This approach allows us to carry out information fusion from multiband image data in a fully integrated way. We have discussed where and when this approach is particularly appropriate.

The

Acknowledgements

Raftery's research was supported by NIH Grant 1R01CA094212-01 and ONR Grant N00014-01-10745. Murtagh acknowledges helpful discussions within the IDHA and MDA projects supported by the French Ministry of Education's ‘Data Masses’ program, and also the European project iAstro, COST Action 283.

References (32)

  • S.G. Djorgovski et al.
  • C. Fraley et al.

    How many clusters? Which clustering method? Answers via model-based cluster analysis

    The Computer Journal

    (1998)
  • C. Fraley et al.

    Model-based clustering, discriminant analysis, and density estimation

    Journal of the American Statistical Association

    (2002)
  • M.H. Hansen et al.

    Model selection and the principle of minimum description length

    Journal of the American Statistical Association

    (2001)
  • R.E. Kass et al.

    Bayes factors

    Journal of the American Statistical Association

    (1995)
  • G. McLachlan et al.

    Mixture Models

    (1988)
  • Cited by (24)

    • Dirichlet Gaussian mixture model: Application to image segmentation

      2011, Image and Vision Computing
      Citation Excerpt :

      Accurately segmented images provide more and useful information for diagnosis and quantitative analysis. However, automated segmentation [1,2] is still a very challenging research topic, due to overlapping intensities and low contrast in images, as well as noise perturbation. In literature, different methodologies proposed for image segmentation include mean shift [3,4], clustering methods [5,6], graph based techniques [7–10], partial differential equations (PDE) based segmentation techniques [11,12], and region competition [13].

    • Bayesian mixture models of variable dimension for image segmentation

      2009, Computer Methods and Programs in Biomedicine
      Citation Excerpt :

      In image segmentation applications, the Bayesian model selection criterion takes into account spatial neighborhood information, and is termed pseudo-likelihood information criterion (PLIC) [7]. However, in this kind of applications model-based Gaussian fitting has been found to be unstable and algorithmically non-robust [1]. Additional correction mechanisms of computational nature have been devised to optimize estimates, such as model-based cluster trees [1].

    View all citing articles on Scopus
    View full text