Elsevier

Signal Processing

Volume 93, Issue 9, September 2013, Pages 2559-2572
Signal Processing

Unsupervised color–texture segmentation based on multiscale quaternion Gabor filters and splitting strategy

https://doi.org/10.1016/j.sigpro.2013.02.010Get rights and content

Abstract

This paper proposes a new method for color–texture segmentation based on a splitting framework with graph cut technique. To process the scale difference of quaternion Gabor filter (QGF) features of a color textured image, a new multiscale QGF (MQGF) is introduced to describe texture attributes of the given image. Then, the segmentation is formulated in terms of energy minimization gradually obtained using binary graph cuts, where color and MQGF features are modeled with a multivariate finite mixture model, and minimum description length (MDL) principle is integrated into this framework as a splitting criterion. In contrast to previous approaches, our method finds an optimal segmentation by balancing energy cost and coding length, and the segmentation result is determined during the splitting process automatically. Experimental results on both synthetic and real natural color textured images demonstrate the good performance of the proposed method.

Highlights

► A new multiscale quaternion Gabor filter is introduced to extract texture features. ► A new Splitting framework is used to find an optimal segmentation. ► The proposed method can decide the segment result in unsupervised way.

Introduction

Color image segmentation is one of the fundamental problems in computer vision and image processing. In general, the segmentation results rely on the appropriate features extracted from color images; the more discriminative power the extracted features are, the more likely the eventual segmentation will be successful [1]. Color and texture are essential features for color images and often used to segment the images with complex characteristics. Existing color image segmentation algorithms can be generally classified into three major families: boundary-based, feature-based and region-based approaches.

Boundary-based approaches [2], [3] usually start with some initial contours enclosing objects or subobjects, followed by a set of shrink and expansion operations to modify the contours until some energy function reaches a minimum. One problem with these approaches is that they are easy to get trapped in local minima. In addition, to obtain the desired global object, numerous interactions are necessary. Feature-based approaches try to classify pixels into compact but well-separated clusters in a feature space without considering their connectivity to similarly classified pixels. Grayscale thresholding [4] and distance-based classification [5] belong to this category. The main drawback of these approaches is that the pixels from disconnected segments of a color image may be clustered together if their feature spaces overlap.

Region-based approaches [6], [7], [8] extend feature-based approaches by trying to preserve the spatial structure and edge information while grouping pixels into multiple consistent regions that satisfy some predefined criteria. Region growing, region merging and region splitting methods are some examples. In region-based approaches, the selection of segmentation criterion is an especially important yet still challenging problem. Hoang et al. [7] used a preselected threshold to merge the regions obtained by K-means in a composite feature space, and showed that using color and texture in combination results in better discrimination than using the color and texture features separately. In [6], Deng and Manjunath proposed an unsupervised color–texture segmentation method (referred to as JSEG) which carries out a region growing procedure with a novel segmentation criterion. One shortcoming of JSEG is that it is easy to generate over-segmented results. A compression-based texture merging (CTM) method for natural image segmentation is proposed by Yang et al. [8]. CTM models the distribution of the color–texture features using a mixture of Gaussian distributions, and utilizes the lossy minimum description length (MDL) [9] principle to merge the over-segmented regions (superpixels) in an unsupervised way.

Recently, graph-based approaches for image segmentation have been developed. One essential feature of these approaches is that the segmentation energy function combines boundary regularization with regional properties. Normalized cuts (Ncuts) proposed by Shi and Malik [10] and Yu and Shi [11] are general segmentation approaches by solving an eigensystem, which can capture intuitively salient parts in an image and are a landmark in current popular spectral clustering research, but it is not perfectly fit for the nature of image segmentation because ad hoc approximations must be introduced to relax the NP-hard computational problem [12], and these approximations are not well understood and often lead to unsatisfactory results. Additionally, the heavy computational cost is a disadvantage rooted in spectral clustering algorithms. Some methods based on nonnegative matrix factorization [13], [14], [15] have been proposed to improve the performance of spectral clustering.

Another popular graph-based approach is graph cut algorithm [16], [17], [18] which is based on Markov random fields (MRFs). This approach is a general purpose interactive segmentation technique, which solves only the two class segmentation problem. In order to obtain the segmentation of multiple regions in an image, some graph cut-based approaches [19] have been developed. However, in these approaches, the number of clusters has to be given at the beginning and cannot be adjusted during segmentation. In addition, the segmentation results are sensitive to this number. To avoid this problem, Chen et al. [12] proposed an iterative maximum a posteriori (MAP) and maximum-likelihood (ML) method to determine the number of segments in an image automatically, but if the region number becomes too large during the iterative process, this method will have a high computational complexity to minimize the MAP-MRF estimation by α-expansion [19]. Another method proposed by Kim and Hong [20] uses binary split moves to find minimum cuts. One drawback of this method is that the one dimension (1-D) texton features may cause much critical information to be lost. In addition, the termination conditions for this method cannot fit for most color textured images.

Due to the multichannel nature of color images, the literature on color image segmentation is not as extensive as that on gray-level image segmentation. Most of the algorithms discussed above fail to couple the information contained in color (e.g., red, green and blue) channels. Recently, quaternion theories and techniques have been employed in color image processing; examples include quaternion Fourier transform (QFT) [21], quaternion principal component analysis (QPCA) [22], quaternion Gabor filter (QGF) [23], [24] and quaternion Zernike moments [25]. Compared with the general segmentation algorithms, the quaternion framework is adopted since it offers the scope to process color images holistically, rather than as separate color space components, and thereby handles the coupling between the color channels naturally.

In this paper, we propose an efficient color–texture segmentation method based on a multiscale quaternion Gabor filter (MQGF) and a splitting framework. The segmentation is formulated as a labeling problem solved by using iterative binary graph cuts and MDL principle. In contrast to previous unsupervised color–texture segmentation approaches, the main contributions of our method are twofold. First, a new MQGF is introduced to extract texture features of color images, which can effectively handle not only the coupling between color channels but the scale difference of QGF features of color textured images. Second, we propose a new splitting framework, in which binary graph cut technique in conjunction with the lossy MDL [9] principle is used to obtain the optimal segmentation automatically.

The rest of this paper is organized as follows. Section 2 describes the basic concepts of quaternion and the MQGF. In Section 3, we focus on the extraction of color and texture features and the mixture model of the composite features. In next section, the proposed segmentation method is explained in detail. Section 5 describes the extensive experiments carried out to test and validate the proposed method, followed by a brief conclusion in Section 6.

Section snippets

Multiscale quaternion Gabor filters

A Hamilton quaternion qH is defined asq=q0+q1i+q2j+q3k,where q0, q1, q2, q3R, and i, j, k are three imaginary numbers. H can be regarded as a 4-D vector space over R with the natural definition of addition and scalar multiplication. In addition, H is made into a ring by the usual distributive law together with the multiplication rules: i2=j2=ijk=1, ij=ji=k. If the scalar and vector parts of a quaternion q are denoted by Sq=q0 and Vq=q1i+q2j+q3k respectively, the product of two quaternions q

Feature extraction

Original RGB color values of a color textured image I are selected as color feature of the image. Then, the color feature of a pixel p is represented by a 3-D vectorFpC=(FpR,FpG,FpB)Twhere FpR, FpG and FpB respectively denote the red, green and blue components of pixel p. Color is one of the most dominant and distinguishable low-level visual features in describing images. However, only using color feature, a large number of color textured images usually cannot be segmented well.

As for texture

Segmentation method

In this section, we describe the proposed color–texture segmentation method. The segmentation is formulated in terms of energy minimization with graph cuts. Based on the model of the composite color and texture features, the segmentation energy consists of a data term and a smoothness term which are used to construct a weighted graph. Unlike previous region-based approaches, our method finds a global optimal solution by a gradually refining process using binary s-t graph cuts. In addition, the

Experimental results

In this section, we test our method with a large set of synthetic and natural images. The first test is performed on a synthetic dataset of mosaic images from the MIT VisTex texture database [33] and the second set of experiments is conducted on the Berkeley database [34] and Weizmann database [35]. To illustrate the validity of the proposed schema, some state-of-the-art color–texture segmentation algorithms are used for comparison. JSEG algorithm [6], a standard color–texture segmentation

Conclusions

In this paper, we have developed an efficient method for unsupervised color–texture segmentation. A new MQGF is first introduced to extract texture attributes of a color textured image, which handles not only the coupling between the channels but also the scale difference of QGF features of the given image efficiently. Then, the MQGF and color features are integrated into a new iterative splitting framework with binary graph cuts and the MDL principle to find optimal segmentation automatically.

Acknowledgments

The authors would like to thank the anonymous reviewers for their insightful suggestions, and Dr. Naiyang Guan for his valuable discussions. This work was supported by the National Natural Science Foundation of China (Grant 60972098).

References (38)

  • D.H. Ballard et al.

    Computer Vision

    (1982)
  • J.T. Tou et al.

    Pattern Recognition Principles

    (1974)
  • Y. Deng et al.

    Unsupervised segmentation of color–texture regions in images and video

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (2001)
  • Y. Ma et al.

    Segmentation of multivariate mixed data via lossy data coding and compression

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (2007)
  • J. Shi et al.

    Normalized cuts and image segmentation

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (2000)
  • S.X. Yu, J. Shi, Multiclass spectral clustering, in: Proceedings of IEEE International Conference on Computer Vision,...
  • S. Chen et al.

    Image segmentation by MAP-ML estimations

    IEEE Transactions on Image Processing

    (2010)
  • N. Guan et al.

    Non-negative patch alignment framework

    IEEE Transactions on Neural Networks

    (2011)
  • N. Guan, D. Tao, Z. Luo, J. Shawe-Taylor, Mahnmf: manhattan non-negative matrix factorization, Arxiv preprint...
  • Cited by (25)

    • An evolutionary single Gabor kernel based filter approach to face recognition

      2017, Engineering Applications of Artificial Intelligence
      Citation Excerpt :

      3D morphological model for FR used an input face image to build a 3D model (Blanz and Vetter, 2003; Paysan et al., 2009). Recently, Gabor filter-based approaches are also extensively used in other image processing areas like texture segmentation, text detection, medical image processing, etc. (Aradhya and Pavithra, 2014; Cruz-Aceves et al., 2016; Li et al., 2013; Raghavendra et al., 2016). The Gabor filter bank has been used as a tool for classification of breast cancer as normal, benign or malignant (Raghavendra et al., 2016).

    • Time–frequency localized three-band biorthogonal wavelet filter bank using semidefinite relaxation and nonlinear least squares with epileptic seizure EEG signal classification

      2017, Digital Signal Processing: A Review Journal
      Citation Excerpt :

      Further, the TFP of the filters of the designed three-band filter bank is better than the two-band biorthogonal filter banks designed by Manish et al. [15]. The designed time–frequency localized filter bank may find applications in image coding [21], content based image retrieval [37], fingerprint recognition [38], discrimination of 1-D signals such as: speech [39], audio signals [40,41] EEG [42], Electrocardiogram (ECG) [43–45] and classification [46–50] and segmentation [51–53] of 2-D signals with low and high frequency content. Epilepsy is a common neurological disorder which is characterized by sudden electrical disturbance in the brain which may lead to the critical situation.

    • Color-texture cosegmentation based on nonlinear compact multi-scale structure tensor and TV-flow

      2017, Signal Processing
      Citation Excerpt :

      For the first category, different construction manners are employed in different approaches. For examples, as in [16,17], the color and texture features are extracted separately, and then a control parameter is introduced to balance the proportion of color and texture to form a higher-dimension color-texture feature vector. While in [18], it extends the modeling way of traditional structure tensor by including the color information (or intensity) with image derivatives to reform the tensor product, and then a higher-order color-texture extended structure tensor is constructed.

    • Research of semantic understanding on target region of interest for fuzzy image

      2015, Engineering Applications of Artificial Intelligence
      Citation Excerpt :

      More research on fuzzy signal processing were achieved (Kılıç and Leblebicioğlu, 2012; Lin et al., 2011; Bailador and Triviño, 2010). But to achieve accurate target recognition, some researches on image processing must be carried out mainly, and these research results are more in number (Zhang and Li, 2011; Berger, 2013; Deng and Xie, 2013; Li et al., 2013). From the analysis on existing information on image processing, it was found, so far, that most processing operators on image were almost some deterministic processing operators that can carry out edge detection, corner detection, etc., such as Canny operator, Gaussian operator, zero-crossing gradient algorithm, and so on.

    • Quaternion Capsule Neural Network With Region Attention for Facial Expression Recognition in Color Images

      2022, IEEE Transactions on Emerging Topics in Computational Intelligence
    View all citing articles on Scopus
    View full text