Image segmentation from scale and rotation invariant texture features from the double dyadic dual-tree complex wavelet transform
Research Highlights
► D3T−CWT provides useful features for improving segmentation. ► Scale and rotation invariant texture features from D3T-CWT can be generated using DFT or circular-correlation approach. ► Sensitivity analysis shows that circular-correlation approach gives best segmentation performance on images with scaled and rotated textures.
Introduction
Automatic image segmentation aims to divide a picture into regions corresponding to objects in the real world. This type of segmentation plays an important role in understanding imagery to the extent that it divides the picture into semantically meaningful components. These components help to describe the content of the image and consequently, segmentation of this type can play an important role in content-based image retrieval applications [1].
The common approach taken in segmentation is to first of all classify the image into regions with common properties, such as texture, pixel colour, pixel intensity and spatial information [2], [3], [4]. A feature vector is formed at all locations in the image (optionally at multiple scales) using those descriptions. A classification process assigns each site to a class that indicates its membership of a region [5]. The accuracy of the segmentation can be evaluated by comparing it to one or more ground truth segmentations generated by human operators [6], [7].
Texture plays an important role in image segmentation. Over the years, Gabor wavelets have been shown to produce good results for segmentation [8], [9]. Research into the human visual system suggests that vision is based on responses of cells with characteristics similar to Gabor wavelets [10], [11]. As such, Gabor wavelets have played an important role in image processing as a representation for texture and have been found to be successful in segmentation [8], [9] and texture image representation for digital libraries [12], [13]. They have been adopted as a texture descriptor for MPEG 7 [14]. Manthalkar, Biswas and Chatterji describe a way of generating scale and rotation invariant 2D Gabor based features by combining them with the Discrete Fourier Transform (DFT) [13]. This combination was used as a feature for texture classification in a digital library that stored single content texture images presented at different scales and rotations.
Model-based techniques have also produced good segmentation results with examples including Gaussian Mixture Models (GMMs) and Markov Random Fields (MRFs). Expectation Maximisation is an effective approach for segmenting images in model-based techniques [5], [15].
The use of complex wavelets for signal analysis has recently become popular through Kingsbury's Dual-Tree Complex Wavelet Transform (DT-CWT) [16]. This is especially useful for texture representation due to the shift invariance of its coefficients and its Gabor-like frequency response in a compact representation. In recent investigations approaches based on complex wavelets were shown to produce accurate segmentation results [17], [18], [19].
Scale and rotation invariant texture features are important concepts in image segmentation. The traditional approaches to classification work through a process of discrimination. The advantage gained from using a scale and rotation invariant texture feature is that the process can associate common texture features regardless of their scale or rotation. For example, in order to segment an image of a zebra from the background, it is necessary for the feature to capture the concept of “stripes” (regardless of scale or direction) and for that feature to be distinguishable from other features. Importantly, scale and orientation provides cues for further stages in post-processing in the human visual system [20]. Applying scale and rotation invariant texture features has also shown to improve query by example performance over standard texture features [21].
A contribution herein is the approach of filtering at and between dyadic scales using the DT-CWT that we term the Double Dyadic DT-CWT (D3T-CWT). We describe a new method for generating scale and rotation invariant texture features from the D3T-CWT for image segmentation. This approach is based on maximising the circular-correlation with a specific mask exhibiting a target configuration. In particular, the results of segmentation using this approach compares well against Gabor wavelets and against features directly derived from the DT-CWT. Previously, we used the D3T-CWT to generate scale and rotation invariant texture features using the magnitude of coefficients from the DFT.
This paper is divided into seven sections. Section 2 describes complex wavelets and the shortcomings of the dyadic based DT-CWT approach for our required application. Section 3 provides an explanation of the D3T-CWT approach. A description of image segmentation using the different approaches for representing texture, as employed in this paper is provided in Section 4. Next, Section 5 details the experimental procedure used to compare the performance of segmentation from the different texture features, while the results of this comparison are presented in Section 6. Finally, Section 7 discusses these results.
Section snippets
Complex wavelets
A complex wavelet transform is a Discrete Wavelet Transform (DWT) that produces complex valued coefficients. A useful property of the complex wavelet transform is that the magnitude of the transformed data is approximately shift invariant. This property makes the complex wavelet transform useful for certain signal analysis applications.
Attributes of the D3T-CWT
The D3T-CWT has the ability to analyse a signal at and between dyadic scales, resulting in two important attributes. The first is the D3T-CWT's improved response to scale change in the input signal. The second is an improved ability to discriminate different signals compared to the DT-CWT.
Image segmentation using texture
Image segmentation aims to group individual pixels into classes that relate to objects in the picture. The input into this classification process can consist of pixel values (such as intensity or colour) or be features derived from the pixel values (such as texture or spatial information).
Herein, texture features are extracted from an image using the following methods:
- •
Gabor wavelets as designed by Manjunath [12]
- •
DT-CWT similar to the approach by Kam [17]
- •
Scale and rotation invariant feature from
Experimental procedure
The experiments conducted were aimed at evaluating the performance of both of our scale and rotation invariant texture features derived from the D3T-CWT when compared with more traditional texture features derived from Gabor wavelets and the DT-CWT. The focus has been on conducting a comprehensive analysis that examines the performance of automatic segmentation against hand segmentations produced by human operators over two large databases of imagery.
Our approach to image segmentation here has
Experimental results
In this section, we examine the performance in automatic image segmentation of the four different methods over the two imagery databases, the Berkeley set and the Corel-SR set. Performance of the methods is judged quantitatively in terms of segmentation accuracy achieved and qualitatively to assess the results visually.
Conclusion
The DT-CWT has proven to be a useful signal analysis tool for image processing. However, it has limitations due to its filtering at only dyadic scales. We have proposed the D3T-CWT as an extension to the DT-CWT to allow signal analysis at and between dyadic scales using filters with overlapping responses. This new transform facilitates the production of scale and rotation invariant texture features for use in image segmentation using the new method presented herein. This approach employs
Acknowledgements
The authors would like to thank Prof Nick Kingsbury from the University of Cambridge for his assistance with the Dual-Tree Complex Wavelet Transform. In particular, Edward Lo would like to thank his colleague, Dr Iain Macleod for his invaluable advice and the continued support from the Defence Science and Technology Organisation in facilitating this research.
References (40)
- et al.
Unsupervised texture segmentation using gabor filters
Pattern Recognit.
(1991) Complex wavelets for shift invariant analysis and filtering of signals
Appl. Comput. Harmon. Anal.
(2001)- et al.
Content-based image retrieval at the end of the early years
IEEE Trans. Pattern Anal. Mach. Intell.
(2000) - et al.
Segmentation of color textures
IEEE Trans. Pattern Anal. Mach. Intell.
(2000) - et al.
A spatial extension of cielab for digital color image reproduction
J. Soc. Inf. Display
(1997) - et al.
Perceptually-tuned multiscale color-texture segmentation
- et al.
Statistical pattern recognition: a review
IEEE Trans. Pattern Anal. Mach. Intell.
(2000) - et al.
A measure for objective evaluation of image segmentation algorithms
- et al.
Unsupervised performance evaluation of image segmentation
EURASIP J. Appl. Signal Process.
(2006) - et al.
Perceptually consistent segmentation of texture using multiple channel filter
Theory of communication
J. Inst. Electron. Eng.
Image representation using 2d gabor wavelets
IEEE Trans. Pattern Anal. Mach. Intell.
Texture features for browsing and retrieval of image data
IEEE Trans. Pattern Anal. Mach. Intell.
Rotation and scale invariant texture classification using gabor wavelets
Colour and texture descriptors
IEEE Trans. Circuits Syst. Video Technol.
Markov random fields and stochastic image models
Scale and rotation invariant texture features from the dual-tree complex wavelet transform
Image segmentation using invariant texture features from the double dyadic dual-tree complex wavelet transform
The Perception of the Visual World
Cited by (34)
Integrating spectral and image data to detect Fusarium head blight of wheat
2020, Computers and Electronics in AgricultureCitation Excerpt :Selesnick et al. (2005) further improved the decomposition and reconstruction algorithm of DTCWT in 2005. It is particularly useful for texture representation because of the mobility invariance of its coefficients and its Gabor-like frequency response in a compact representation (Lo et al., 2011). DTCWT has been widely used for image processing and can be executed separately to generate a quadtree structure, with two trees for the image rows and two trees for the image columns (Yang et al., 2018).
An automated method for tree-ring delineation based on active contours guided by DT-CWT complex coefficients in photographic images: Application to Abies alba wood slice images
2015, Computers and Electronics in AgricultureCitation Excerpt :We describe and test a new method to delineate successive concentric tree-rings in Abies alba flatbed-scanned images by running successive active contour delineating tree-rings and inter tree-rings alternately. In matters of texture image analysis and synthesis, the Q-Shift Dual-Tree Complex Wavelet Transform (DT-CWT) (Kingsbury, 2001), originating from the wavelet community, has proven to be superior to all other wavelet transforms (De Rivaz, 2000; Lo et al., 2011). The DT-CWT proposed by Kingsbury in (Kingsbury, 2001) is a relatively recent enhancement of the Discrete Wavelet Transform (DWT) where the DWT is considered to be one of the most useful mathematical tools for multi-resolution image analysis (Mallat, 1989).
A scale- and orientation-adaptive extension of Local Binary Patterns for texture classification
2015, Pattern RecognitionCitation Excerpt :A general drawback of this class of methods is that scaling can only be compensated at dyadic steps. As an improvement, Lo et al. [4] use a Double-Dyadic DT-CWT combined with a Discrete Fourier Transform (DFT) to construct scale-invariant feature descriptors at sub-dyadic scales. The periodicity of the DFT is also exploited by Riaz et al. [5] to compute scale-invariant features by compensating the shifts in accumulated Gabor filter responses.
A color extraction algorithm by segmentation
2023, Scientific ReportsQSurfNet: a hybrid quantum convolutional neural network for surface defect recognition
2023, Quantum Information ProcessingState of the Art in Defect Detection Based on Machine Vision
2022, International Journal of Precision Engineering and Manufacturing - Green Technology
- 1
Employed by the Defence Science and Technology Organisation, Canberra, Australia.