Elsevier

Pattern Recognition Letters

Volume 30, Issue 15, 1 November 2009, Pages 1404-1412
Pattern Recognition Letters

Color image segmentation using an enhanced Gradient Network Method

https://doi.org/10.1016/j.patrec.2009.07.005Get rights and content

Abstract

The objective of this paper is to evaluate a new combined approach intended for reliable color image segmentation, in particular images presenting color structures with strong but continuous color or luminosity changes, such as commonly found in outdoors scenes. The approach combines an enhanced version of the Gradient Network 2, with common region-growing approaches used as pre-segmentation steps. The GNM2 is an post-segmentation procedure based on graph analysis of global color and luminosity gradients in conjunction with a segmentation algorithm to produce a reliable segmentation result. The approach was automatically evaluated using a close/open world approach. Two different region-growing segmentation methods, CSC and Mumford and Shah with and without the GNM post-processing were compared against ground truth images using segmentation evaluation indices Rand and Bipartite Graph Matching. These results were also confronted with other well established segmentation methods (RHSEG, Watershed, EDISON, JSEG and Blobworld).

Introduction

Natural color scenes, such as outdoors images composed by many colored objects that are acquired under uncontrolled conditions can show complex illumination patterns across one same object in the picture. Examples are variations in lightness and specular effects. State-of-the-art region-growing segmentation methods present two main features that limit their applicability for dealing efficiently with natural scenes:

  • (a)

    A static region similarity concept: where pixels or textures within a region are expected to be homogeneous. Typical natural scenes, however, show strong continuous variations of color, presenting a different, dynamic order that is not taken into account by such algorithms. They divide a sky region with different intensities of blue into segments or will represent an irregularly illuminated surface as a set of different regions. When the parameters of such algorithms are stressed in order to try to accomplish a correct segmentation of a large object showing a long continuous gradient of color, typically with a gradual but large color variation, a region leakage into other objects in the image is likely to occur. Then the algorithm will become unstable and even inapplicable;

  • (b)

    Increase in complexity to present more stable results: which usually demands complex computations to detect segment-correlation clues, or are built upon additional texture information. This slows down considerably the processing time without being much more stable when extreme color variations are present.

A number of segmentation approaches already tried to cope with one or both problems: Rehrmann and Priese (1998) propose a hierarchical model to improve the segmentation of color scenes, while Deng and Manjunath (2001), Dupuis and Vasseur (2006), Kato and Pong (2006) address the problem by the additional analysis of object textures. Some authors, as Dony and Wesolkowski, 1999, Schneider et al., 2000, have tried to overcome the problem using different methods of chromatic and luminescence evaluation. Finally, Klinker et al. (1990), Tsang and Tsang (1996), Healey (1992) proposed illumination models for the formulation of segmentation algorithms.

We addressed this problem before, developing the Gradient Network Method (GNM) (von Wangenheim et al., 2007). GNM was originally developed intending to solve the problem of luminance and reflection variations by searching for structured gradients along the various neighboring segments, providing a fast and reliable post-segmentation approach for images otherwise difficult to segment (von Wangenheim et al., 2008). The method was also developed to be used as a framework, where different structured gradient analysis strategies could be embedded. The conceptual ideas, the general philosophy underlying the GNM algorithm and a detailed algorithmic description of the GNM are detailed in (von Wangenheim et al., 2007, von Wangenheim et al., 2008).

Our purpose in this paper is to present an enhanced version of this algorithm with some new features, which were included into the method following the intended framework philosophy. The new heuristics try to improve the results while still keeping a similar approach towards the problem of dealing with illumination effects. As its former version, it works together with a segmentation algorithm, which provides a pre-segmentation as a starting point. The general approach is to rely on segmentation algorithms such as CSC (Rehrmann and Priese, 1998) or any other producing as output an over-segmentation with edge preservation and subsequently apply our algorithm, which will iteratively merge segments logically connected through organized gradient patterns improving the final segmentation. Besides, in this paper we also provide objective quantitative segmentation validation results through ground-truth-based segmentation quality measures, which were not present in our former work, where we only discussed our results qualitatively.

Section snippets

Objectives

In our former papers, we presented the GNM approach, discussed that it was developed intending it to be used as a framework for different post processing heuristics (von Wangenheim et al., 2007) and offered a set of results that were empirically evaluated from the performance point of view (von Wangenheim et al., 2008). In this paper we pursued two main objectives:

  • 1.

    Objective segmentation quality validation: for this purpose, we extended the qualitative approach used in (von Wangenheim et al.,

Enhancing the Gradient Network Method segmentation with additional heuristics: description of the new features

The Gradient Network Method previously described in von Wangenheim et al. (2007) was developed to deal with segmentation problems where objects in the scene will be represented by several different but similar and gradually varying color shades, as they often are found in outdoors scenes, as depicted in Fig. 1. The GNM looks for a higher degree of organization in the structure of the scene through search and identification of continuous and smooth color gradients. Following these same goals and

Experiment

In order to be able to objectively evaluate we have proposed an objective and quantitative evaluation of GNM2 approach. Then, to choose the optimal segmentation parameters, we accepted that ground-truths (GTs), or hand-made segmentations, representing the judgment of a human observer, should play the role of golden standards. This process of evaluation of image segmentation results was investigated by a number of researchers (Unnikrishnan et al., 2007, Sahasrabundhe et al., 1999, Di Gesu and

Results

The GNM and GNM2 combined segmentation results in general obtained a better average score with both quality measures pixel counting and set matching (see Table 1). The GNM2 together with the Mumford and Shah algorithm as its pre-segmentation step presented the best results, with a mean Rand index of 0.1177 and a mean BGM index of 0.1678. It also showed the lowest mean variances and mean standard deviations for both Rand and BGM indexes. These last results show that the GNM2 algorithm was also

Discussion

We have shown through well known validation measures that the quality of the segmentations generated by our two-step approach is very promising and comparable to segmentations generated by state-of-the-art segmentation methods that were available for comparison when this paper was being written. The Gradient Network Method and its new version are segmentation post-processing methods that are independent of the region-growing method applied to generate the super-segmented input image, as shown

Acknowledgement

D.D. Abdala enjoys a CNPq-Brazil Ph.D scholarship under the process No. 290101/2006-9.

References (31)

  • EDISON image segmentation and edge detector, 2007. <http://www.caip.rutgers.edu/riul/research/code/EDISON/index.html>....
  • G. Healey

    Segmenting images using normalized color

    IEEE Trans. Syst. Man Cybernetics

    (1992)
  • X. Jiang et al.

    Distance measures for image segmentation evaluation

    EURASIP J. Appl. Signal Process.

    (2006)
  • G. Klinker et al.

    A physical approach to color image understanding

    Int. J. Comput. Vision.

    (1990)
  • Martin, D., 2002. An Empirical Approach to Grouping and Segmentation. PhD dissertation. University of California,...
  • Cited by (0)

    View full text