Color image segmentation using an enhanced Gradient Network Method
Introduction
Natural color scenes, such as outdoors images composed by many colored objects that are acquired under uncontrolled conditions can show complex illumination patterns across one same object in the picture. Examples are variations in lightness and specular effects. State-of-the-art region-growing segmentation methods present two main features that limit their applicability for dealing efficiently with natural scenes:
- (a)
A static region similarity concept: where pixels or textures within a region are expected to be homogeneous. Typical natural scenes, however, show strong continuous variations of color, presenting a different, dynamic order that is not taken into account by such algorithms. They divide a sky region with different intensities of blue into segments or will represent an irregularly illuminated surface as a set of different regions. When the parameters of such algorithms are stressed in order to try to accomplish a correct segmentation of a large object showing a long continuous gradient of color, typically with a gradual but large color variation, a region leakage into other objects in the image is likely to occur. Then the algorithm will become unstable and even inapplicable;
- (b)
Increase in complexity to present more stable results: which usually demands complex computations to detect segment-correlation clues, or are built upon additional texture information. This slows down considerably the processing time without being much more stable when extreme color variations are present.
A number of segmentation approaches already tried to cope with one or both problems: Rehrmann and Priese (1998) propose a hierarchical model to improve the segmentation of color scenes, while Deng and Manjunath (2001), Dupuis and Vasseur (2006), Kato and Pong (2006) address the problem by the additional analysis of object textures. Some authors, as Dony and Wesolkowski, 1999, Schneider et al., 2000, have tried to overcome the problem using different methods of chromatic and luminescence evaluation. Finally, Klinker et al. (1990), Tsang and Tsang (1996), Healey (1992) proposed illumination models for the formulation of segmentation algorithms.
We addressed this problem before, developing the Gradient Network Method (GNM) (von Wangenheim et al., 2007). GNM was originally developed intending to solve the problem of luminance and reflection variations by searching for structured gradients along the various neighboring segments, providing a fast and reliable post-segmentation approach for images otherwise difficult to segment (von Wangenheim et al., 2008). The method was also developed to be used as a framework, where different structured gradient analysis strategies could be embedded. The conceptual ideas, the general philosophy underlying the GNM algorithm and a detailed algorithmic description of the GNM are detailed in (von Wangenheim et al., 2007, von Wangenheim et al., 2008).
Our purpose in this paper is to present an enhanced version of this algorithm with some new features, which were included into the method following the intended framework philosophy. The new heuristics try to improve the results while still keeping a similar approach towards the problem of dealing with illumination effects. As its former version, it works together with a segmentation algorithm, which provides a pre-segmentation as a starting point. The general approach is to rely on segmentation algorithms such as CSC (Rehrmann and Priese, 1998) or any other producing as output an over-segmentation with edge preservation and subsequently apply our algorithm, which will iteratively merge segments logically connected through organized gradient patterns improving the final segmentation. Besides, in this paper we also provide objective quantitative segmentation validation results through ground-truth-based segmentation quality measures, which were not present in our former work, where we only discussed our results qualitatively.
Section snippets
Objectives
In our former papers, we presented the GNM approach, discussed that it was developed intending it to be used as a framework for different post processing heuristics (von Wangenheim et al., 2007) and offered a set of results that were empirically evaluated from the performance point of view (von Wangenheim et al., 2008). In this paper we pursued two main objectives:
- 1.
Objective segmentation quality validation: for this purpose, we extended the qualitative approach used in (von Wangenheim et al.,
Enhancing the Gradient Network Method segmentation with additional heuristics: description of the new features
The Gradient Network Method previously described in von Wangenheim et al. (2007) was developed to deal with segmentation problems where objects in the scene will be represented by several different but similar and gradually varying color shades, as they often are found in outdoors scenes, as depicted in Fig. 1. The GNM looks for a higher degree of organization in the structure of the scene through search and identification of continuous and smooth color gradients. Following these same goals and
Experiment
In order to be able to objectively evaluate we have proposed an objective and quantitative evaluation of GNM2 approach. Then, to choose the optimal segmentation parameters, we accepted that ground-truths (GTs), or hand-made segmentations, representing the judgment of a human observer, should play the role of golden standards. This process of evaluation of image segmentation results was investigated by a number of researchers (Unnikrishnan et al., 2007, Sahasrabundhe et al., 1999, Di Gesu and
Results
The GNM and GNM2 combined segmentation results in general obtained a better average score with both quality measures pixel counting and set matching (see Table 1). The GNM2 together with the Mumford and Shah algorithm as its pre-segmentation step presented the best results, with a mean Rand index of 0.1177 and a mean BGM index of 0.1678. It also showed the lowest mean variances and mean standard deviations for both Rand and BGM indexes. These last results show that the GNM2 algorithm was also
Discussion
We have shown through well known validation measures that the quality of the segmentations generated by our two-step approach is very promising and comparable to segmentations generated by state-of-the-art segmentation methods that were available for comparison when this paper was being written. The Gradient Network Method and its new version are segmentation post-processing methods that are independent of the region-growing method applied to generate the super-segmented input image, as shown
Acknowledgement
D.D. Abdala enjoys a CNPq-Brazil Ph.D scholarship under the process No. 290101/2006-9.
References (31)
- et al.
Color image segmentation: Advances and prospects
Pattern Recognit.
(2001) - et al.
Distance-based functions for image comparison
Pattern Recognit. Lett.
(1999) - et al.
Image segmentation by cue selection and integration
Image Video Comput.
(2006) - et al.
A Markov random field image segmentation model for color textured images
Image Vision Comput.
(2006) - et al.
Combining local belief from low-level primitives for perceptual grouping
Pattern Recognit.
(2008) - et al.
Fast approximate energy minimization via graph cuts
IEEE Trans. Pattern Anal. Mach. Intell.
(2001) - et al.
Blobworld: Image segmentation using expectation–maximization and its application to image querying
IEEE Trans. Pattern Anal. Mach. Intell.
(2002) - et al.
Mean shift: A robust approach toward feature space analysis
IEEE Trans. Pattern Anal. Mach. Intell.
(2002) - et al.
Unsupervised segmentation of color texture regions in images and video
IEEE Trans. Pattern Anal. Mach. Intell.
(2001) - Dony, R.D., Wesolkowski, S.B., 1999. Edge detection on color image using RGB vector angles. In: Proc. 1999 IEEE...