Self-supervised texture segmentation using complementary types of features

Abstract

A two-stage texture segmentation approach is proposed where an initial segmentation map is obtained through unsupervised clustering of multiresolution simultaneous autoregressive (MRSAR) features and is followed by self-supervised classification of wavelet features. The regions of “high confidence” and “low confidence” are identified based on the MRSAR segmentation result using multilevel morphological erosion. The second-stage classifier is trained by the “high-confidence” samples and is used to reclassify only the “low-confidence” pixels. The proposed approach leverages on the advantages of both MRSAR and wavelet features. Experimental results show that the misclassification error can be significantly reduced by using complementary types of texture features.

Introduction

Texture is a fundamental characteristic of natural images that, in addition to color, plays an important role in human visual perception and provides information for image understanding and scene interpretation. A large volume of research over the past three decades has addressed the problem of texture segmentation and has produced a number of review articles and comparative studies [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12]. Unsupervised image segmentation may be defined as the problem of separating image regions of uniform texture without prior knowledge of the texture types. Texture segmentation does not pose the same problem as texture differentiation or classification, where a supervised system is trained to differentiate between known textures. Texture differentiation is useful in constrained environments where a limited number of textures is encountered; however, the number of textures in natural images is huge and, in general, the types of textures present in a particular image are not known a priori.

Much of the texture segmentation work has concentrated on extracting features that are suitable for texture modeling, followed by feature clustering or classification so that image regions of uniform texture may be identified [13]. Classic texture features include those derived from Laws filters [14], co-occurrence matrices (CO) [15], [16], and cortex transform modulation functions (CTMF) [17]. More recently, a number of new texture features have been considered for texture analysis and segmentation, including multiresolution simultaneous autoregressive (MRSAR) models [18], [19], [20], Markov random field (MRF) models [21], [22], [23], [24] Gabor filters [25], [26], wavelet coefficients [27], [28], [29], [30], and fractal dimension [21], [31]. The results of comparing the relative merits of the different types of features have been nonconclusive and a clear winner has not emerged in all cases [12]. In general, each set of texture features offers unique advantages but also has limitations compared to other feature types.

Meanwhile, one common problem in using feature clustering for image segmentation is that noise in the extracted features may result in misclassification, which takes the form of holes and other fragments [32]. When trying to minimize the problem due to noisy features, another interesting yet challenging problem in texture segmentation is encountered, which may be described as the boundary effect. It usually appears as inaccurate segmentation of boundaries or superfluous narrow regions at the boundary between two textures [18], [32]. It is conjectured that the boundary effect is caused by misclassification when the trajectory of the feature vectors makes a transition through feature space [26]. To make matters worse, the misclassification may be interpreted as a third texture, depending on the nature of the features for a particular image.

The use of multiple types of features in texture segmentation may be exploited to take full advantage of the strengths of each feature type and alleviate some of the problems, such as the boundary ambiguity encountered when the segmentation is based on a single feature type. Ideally, one would like to select complementary types of features that are not highly correlated and, when combined, can improve the final segmentation result [10]. One method of combining two types of features is to concatenate the feature vectors and perform the classification step on the augmented feature set. This method has been used for combining texture and color features [33] as well as texture features of different types [10]. When this approach is taken, one has to worry about normalization issues and the relative weighting of the features in each feature set, since in most cases the number of parameters in the different feature sets is not equal. In addition, by concatenating all the features into a single vector, the segmentation result is based on a global clustering criterion, even though it may be more suitable to constrain clustering of certain features to a local neighborhood.

A second approach in combining different types of features is to perform a segmentation for each of the feature sets independently, and then combine the segmentation maps into one. The difficulty with this approach is coming up with a selection rule for assigning the appropriate segmentation labels to the final segmentation result, where segmentation maps disagree with each other. The simple rule of assuming that there is a texture boundary in the final segmentation map everywhere that a texture boundary appears in any of the individual segmentation maps does not work well because it results in over-segmentation.

In this paper, a two-stage approach to texture segmentation is proposed where, after the initial segmentation, potential problem regions are identified and reclassified through a second stage of refinement. The first stage of segmentation is based on MRSAR features and a global clustering criterion. The second stage is based on self-supervised clustering of wavelet coefficients applied on the regions of low confidence of the segmentation map. The proposed method can be considered as a novel way of combining different types of texture features. In the next section, the initial segmentation method is described, where an improved way of normalizing MRSAR features is utilized for improved results. In Section 3, a method for extracting the segmentation confidence map using morphological operations is discussed. The segmentation refinement process based on wavelet features is presented in Section 4. Finally, 5 Discussion and conclusions, 6 Summary include experimental results and discussions, respectively.