Image segmentation from scale and rotation invariant texture features from the double dyadic dual-tree complex wavelet transform

https://doi.org/10.1016/j.imavis.2010.08.004Get rights and content

Abstract

A goal of image segmentation is to divide an image into regions that have some semantic meaning. Because regions of semantic meaning often include variations in colour and intensity, various segmentation algorithms that use multi-pixel textures have been developed. A challenge for these algorithms is to incorporate invariance to rotation and changes in scale. In this paper, we propose a new scale and rotation invariant, texture-based segmentation algorithm, that performs feature extraction using the Dual-Tree Complex Wavelet Transform (DT-CWT). The DT-CWT is used to analyse a signal at, and between, dyadic scales. The performance of image segmentation using this new method is compared with existing techniques over different imagery databases with operator produced ground truth data. Compared with previous algorithms, our segmentation results show that the new texture feature is capable of performing well over general images and particularly well over images containing objects with scaled and rotated textures.

Research Highlights

D3TCWT provides useful features for improving segmentation. ► Scale and rotation invariant texture features from D3T-CWT can be generated using DFT or circular-correlation approach. ► Sensitivity analysis shows that circular-correlation approach gives best segmentation performance on images with scaled and rotated textures.

Introduction

Automatic image segmentation aims to divide a picture into regions corresponding to objects in the real world. This type of segmentation plays an important role in understanding imagery to the extent that it divides the picture into semantically meaningful components. These components help to describe the content of the image and consequently, segmentation of this type can play an important role in content-based image retrieval applications [1].

The common approach taken in segmentation is to first of all classify the image into regions with common properties, such as texture, pixel colour, pixel intensity and spatial information [2], [3], [4]. A feature vector is formed at all locations in the image (optionally at multiple scales) using those descriptions. A classification process assigns each site to a class that indicates its membership of a region [5]. The accuracy of the segmentation can be evaluated by comparing it to one or more ground truth segmentations generated by human operators [6], [7].

Texture plays an important role in image segmentation. Over the years, Gabor wavelets have been shown to produce good results for segmentation [8], [9]. Research into the human visual system suggests that vision is based on responses of cells with characteristics similar to Gabor wavelets [10], [11]. As such, Gabor wavelets have played an important role in image processing as a representation for texture and have been found to be successful in segmentation [8], [9] and texture image representation for digital libraries [12], [13]. They have been adopted as a texture descriptor for MPEG 7 [14]. Manthalkar, Biswas and Chatterji describe a way of generating scale and rotation invariant 2D Gabor based features by combining them with the Discrete Fourier Transform (DFT) [13]. This combination was used as a feature for texture classification in a digital library that stored single content texture images presented at different scales and rotations.

Model-based techniques have also produced good segmentation results with examples including Gaussian Mixture Models (GMMs) and Markov Random Fields (MRFs). Expectation Maximisation is an effective approach for segmenting images in model-based techniques [5], [15].

The use of complex wavelets for signal analysis has recently become popular through Kingsbury's Dual-Tree Complex Wavelet Transform (DT-CWT) [16]. This is especially useful for texture representation due to the shift invariance of its coefficients and its Gabor-like frequency response in a compact representation. In recent investigations approaches based on complex wavelets were shown to produce accurate segmentation results [17], [18], [19].

Scale and rotation invariant texture features are important concepts in image segmentation. The traditional approaches to classification work through a process of discrimination. The advantage gained from using a scale and rotation invariant texture feature is that the process can associate common texture features regardless of their scale or rotation. For example, in order to segment an image of a zebra from the background, it is necessary for the feature to capture the concept of “stripes” (regardless of scale or direction) and for that feature to be distinguishable from other features. Importantly, scale and orientation provides cues for further stages in post-processing in the human visual system [20]. Applying scale and rotation invariant texture features has also shown to improve query by example performance over standard texture features [21].

A contribution herein is the approach of filtering at and between dyadic scales using the DT-CWT that we term the Double Dyadic DT-CWT (D3T-CWT). We describe a new method for generating scale and rotation invariant texture features from the D3T-CWT for image segmentation. This approach is based on maximising the circular-correlation with a specific mask exhibiting a target configuration. In particular, the results of segmentation using this approach compares well against Gabor wavelets and against features directly derived from the DT-CWT. Previously, we used the D3T-CWT to generate scale and rotation invariant texture features using the magnitude of coefficients from the DFT.

This paper is divided into seven sections. Section 2 describes complex wavelets and the shortcomings of the dyadic based DT-CWT approach for our required application. Section 3 provides an explanation of the D3T-CWT approach. A description of image segmentation using the different approaches for representing texture, as employed in this paper is provided in Section 4. Next, Section 5 details the experimental procedure used to compare the performance of segmentation from the different texture features, while the results of this comparison are presented in Section 6. Finally, Section 7 discusses these results.

Section snippets

Complex wavelets

A complex wavelet transform is a Discrete Wavelet Transform (DWT) that produces complex valued coefficients. A useful property of the complex wavelet transform is that the magnitude of the transformed data is approximately shift invariant. This property makes the complex wavelet transform useful for certain signal analysis applications.

Attributes of the D3T-CWT

The D3T-CWT has the ability to analyse a signal at and between dyadic scales, resulting in two important attributes. The first is the D3T-CWT's improved response to scale change in the input signal. The second is an improved ability to discriminate different signals compared to the DT-CWT.

Image segmentation using texture

Image segmentation aims to group individual pixels into classes that relate to objects in the picture. The input into this classification process can consist of pixel values (such as intensity or colour) or be features derived from the pixel values (such as texture or spatial information).

Herein, texture features are extracted from an image using the following methods:

  • Gabor wavelets as designed by Manjunath [12]

  • DT-CWT similar to the approach by Kam [17]

  • Scale and rotation invariant feature from

Experimental procedure

The experiments conducted were aimed at evaluating the performance of both of our scale and rotation invariant texture features derived from the D3T-CWT when compared with more traditional texture features derived from Gabor wavelets and the DT-CWT. The focus has been on conducting a comprehensive analysis that examines the performance of automatic segmentation against hand segmentations produced by human operators over two large databases of imagery.

Our approach to image segmentation here has

Experimental results

In this section, we examine the performance in automatic image segmentation of the four different methods over the two imagery databases, the Berkeley set and the Corel-SR set. Performance of the methods is judged quantitatively in terms of segmentation accuracy achieved and qualitatively to assess the results visually.

Conclusion

The DT-CWT has proven to be a useful signal analysis tool for image processing. However, it has limitations due to its filtering at only dyadic scales. We have proposed the D3T-CWT as an extension to the DT-CWT to allow signal analysis at and between dyadic scales using filters with overlapping responses. This new transform facilitates the production of scale and rotation invariant texture features for use in image segmentation using the new method presented herein. This approach employs

Acknowledgements

The authors would like to thank Prof Nick Kingsbury from the University of Cambridge for his assistance with the Dual-Tree Complex Wavelet Transform. In particular, Edward Lo would like to thank his colleague, Dr Iain Macleod for his invaluable advice and the continued support from the Defence Science and Technology Organisation in facilitating this research.

References (40)

  • A.K. Jain et al.

    Unsupervised texture segmentation using gabor filters

    Pattern Recognit.

    (1991)
  • N. Kingsbury

    Complex wavelets for shift invariant analysis and filtering of signals

    Appl. Comput. Harmon. Anal.

    (2001)
  • A.W.M. Smeulders et al.

    Content-based image retrieval at the end of the early years

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2000)
  • M. Mirmehdi et al.

    Segmentation of color textures

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2000)
  • X. Zhang et al.

    A spatial extension of cielab for digital color image reproduction

    J. Soc. Inf. Display

    (1997)
  • J.Q. Chen et al.

    Perceptually-tuned multiscale color-texture segmentation

  • A.K. Jain et al.

    Statistical pattern recognition: a review

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2000)
  • R. Unnikrishnan et al.

    A measure for objective evaluation of image segmentation algorithms

  • S. Chabrier et al.

    Unsupervised performance evaluation of image segmentation

    EURASIP J. Appl. Signal Process.

    (2006)
  • N. Zhang et al.

    Perceptually consistent segmentation of texture using multiple channel filter

  • D. Gabor

    Theory of communication

    J. Inst. Electron. Eng.

    (1946)
  • T.S. Lee

    Image representation using 2d gabor wavelets

    IEEE Trans. Pattern Anal. Mach. Intell.

    (1996)
  • B.S. Manjunath et al.

    Texture features for browsing and retrieval of image data

    IEEE Trans. Pattern Anal. Mach. Intell.

    (1996)
  • R. Manthalkar et al.

    Rotation and scale invariant texture classification using gabor wavelets

  • B.S. Manjunath et al.

    Colour and texture descriptors

    IEEE Trans. Circuits Syst. Video Technol.

    (2001)
  • C.A. Bouman

    Markov random fields and stochastic image models

  • A. H. Kam. A General Multiscale Scheme for Unsupervised Image Segmentation. PhD thesis, Department of Engineering,...
  • E.H.S. Lo et al.

    Scale and rotation invariant texture features from the dual-tree complex wavelet transform

  • E.H.S. Lo et al.

    Image segmentation using invariant texture features from the double dyadic dual-tree complex wavelet transform

  • J.J. Gibson

    The Perception of the Visual World

    (1950)
  • Cited by (34)

    • Integrating spectral and image data to detect Fusarium head blight of wheat

      2020, Computers and Electronics in Agriculture
      Citation Excerpt :

      Selesnick et al. (2005) further improved the decomposition and reconstruction algorithm of DTCWT in 2005. It is particularly useful for texture representation because of the mobility invariance of its coefficients and its Gabor-like frequency response in a compact representation (Lo et al., 2011). DTCWT has been widely used for image processing and can be executed separately to generate a quadtree structure, with two trees for the image rows and two trees for the image columns (Yang et al., 2018).

    • An automated method for tree-ring delineation based on active contours guided by DT-CWT complex coefficients in photographic images: Application to Abies alba wood slice images

      2015, Computers and Electronics in Agriculture
      Citation Excerpt :

      We describe and test a new method to delineate successive concentric tree-rings in Abies alba flatbed-scanned images by running successive active contour delineating tree-rings and inter tree-rings alternately. In matters of texture image analysis and synthesis, the Q-Shift Dual-Tree Complex Wavelet Transform (DT-CWT) (Kingsbury, 2001), originating from the wavelet community, has proven to be superior to all other wavelet transforms (De Rivaz, 2000; Lo et al., 2011). The DT-CWT proposed by Kingsbury in (Kingsbury, 2001) is a relatively recent enhancement of the Discrete Wavelet Transform (DWT) where the DWT is considered to be one of the most useful mathematical tools for multi-resolution image analysis (Mallat, 1989).

    • A scale- and orientation-adaptive extension of Local Binary Patterns for texture classification

      2015, Pattern Recognition
      Citation Excerpt :

      A general drawback of this class of methods is that scaling can only be compensated at dyadic steps. As an improvement, Lo et al. [4] use a Double-Dyadic DT-CWT combined with a Discrete Fourier Transform (DFT) to construct scale-invariant feature descriptors at sub-dyadic scales. The periodicity of the DFT is also exploited by Riaz et al. [5] to compute scale-invariant features by compensating the shifts in accumulated Gabor filter responses.

    • State of the Art in Defect Detection Based on Machine Vision

      2022, International Journal of Precision Engineering and Manufacturing - Green Technology
    View all citing articles on Scopus
    1

    Employed by the Defence Science and Technology Organisation, Canberra, Australia.

    View full text