A compact multi-pattern encoding descriptor for texture classification

Abstract

Binary pattern family is considered as a powerful tool for visual texture classification. Most popular methods improve the classification performance by multi-feature fusion. However, many sub-features are redundant and low-discriminative and the classification system has high computational complexity and unsatisfactory results. To handle above problems, this paper proposes a compact multi-pattern encoding descriptor for visual texture classification. First, we develop local extremum patterns and local center pattern to represent the neighborhood intensity changes. Then, we design a compact encoding scheme to encode local maximum, minimum and center patterns into a three-bit binary code, named MMC pattern. Finally, a compact multi-pattern encoding descriptor is proposed by combining the traditional local sign pattern and MMC pattern. Experimental results on five representative texture databases demonstrate that our method achieves the state-of-the-art texture classification performance.

Introduction

Texture classification, as one of the most important tasks in pattern recognition and image processing, has attracted much attention and enjoys a wide range of applications such as medical imaging [1], remote sensing [2], [3], signature verification [4], script identification [5], and face analysis [6], [7], [8]. A successful texture feature should have a good tradeoff between high computational efficiency and rich information representation. Although it has been extensively studied over last few decades, it is still a challenging and open problem to effectively define and represent texture.

In 2017, Liu et al. [9] proposed a taxonomy of binary family and analyzed the merits and demerits of different LBP variants. However, it still does not have a clear definition for texture, and the distinction among different categories is unsatisfactory. Antonio et al. [10] provided a crisp mathematical definition about texture and proposed the histograms of equivalent patterns (HEP) framework for texture analysis. In the HEP framework, the encoding schemes are defined as different “kernel functions” of pixel values. As one of the most basic “kernel functions”, the local binary pattern (LBP) [11] has drawn great attention due to its powerful discriminative capability, computational simplicity, and ease of implementation. However, the traditional LBP only encodes the local sign components of local differences but ignores other local structure information. The incomplete information representation significantly limits its application.

Recent studies suggest that combining complementary sub-features is an effective solution for providing comprehensive texture representation. For example, the authors of [12] proposed a completed modeling of local binary pattern, named CLBP, by combining local sign, magnitude, and center pixel. Due to its good complementary in local information representation, CLBP has rapidly derived many variants, such as CLBC [13] and multi-scale CLBP [14]. The authors of [15] combined the modified color histogram and diagonally symmetric co-occurrence texture pattern for texture and natural scene retrieval. Bnaerjee et al. [16] proposed a local neighborhood intensity pattern for content-based image retrieval. Besides, the authors of [17] found a single but optimal scale for binary pattern descriptors and proposed the scale-adaptive LBP (SALBP) for texture classification. Afterward, in [18], a locally directional and extremal pattern was developed to exploit the information of local texture intensities and directionality effectively. Srishti et al. [19] proposed a local directional peak valley binary pattern for texture retrieval. The authors of [20] introduced a concave and convex binary thresholding function and proposed the local concave and convex micro-structures pattern for texture classification. To achieve a good tradeoff between simplicity and performance, the authors of [21] introduced a cross-scale joint feature representation and proposed the cross-complementary LBP. In [22], four doublets around center pixel were considered to generate the attractive and repulsive center-symmetric LBP that represented the fine and coarse texture information effectively and achieved remarkable performance improvements. In addition, 2D-LBP [23] was proposed by Bin that extracted the spatial contextual information and achieved satisfactory classification results.

The deep learning method is much different from the traditional hand-crafted method in terms of the feature representation. Some researchers have paid much attention to deep texture features generated by convolution neural networks (CNN) and its variants [24], [25], [26], [27]. In particular, Nruyen et al. [28] proposed a handcrafted normalized convolution network (NmzNet) for texture classification. Zhang et al. [29] presented a deep texture encoding network (Deep-TEN) for material recognition. Ayan et al. [30] developed a deep hashing framework for texture retrieval. Despite the success, the deep features still have a series of original issues. On one hand, texture feature distributions require a spatially invariant representation rather than the concatenation of fully connected layer in CNNs [30]. On the other hand, deep models require large amount of annotated training data. However, some texture datasets cannot provide sufficient labeled data. It is worth noting that transfer learning has been used for improving the problems with limited data in texture analysis. Specifically, Asaad et al. [26] applied transfer learning to classify scaled texture patterns. Alessandra et al. [31] used transfer learning and fine-tuning of pretrained models for plankton classification, Song et al. [32] designed a locally transferred fisher vector method for texture classification, which transformed the CNN-based FV descriptors to obtain more discriminative texture representation. Nonetheless, inborn drawbacks, including high computational complex and difficult parameters tuning, make the deep features-based texture representation still controversial. The authors of [33] provided comparative evaluation of handcrafted features vs. CNN-based features. It is interesting to notice that the handcrafted features are still better than CNN-based features for regular texture easily modeled or under little intra-class variation. Therefore, the main research of this paper focuses on the traditional handcrafted binary family features.

Despite the success of LBP variants, there are still significant disadvantages: all these classification systems divide the local neighborhood into different sub-features to represent local texture from different aspects. But these sub-features are usually not completely rotation invariant. In addition, most of sub-features are low discriminative and have high dimensionality, which make the classification system more complex and redundant. Obviously, these LBP-variants improve the classification accuracy and robustness at the price of high computational complexity that, in turn, severely limits their application in many real-world tasks. Despite recent achievements of LBP-variants, there is still a large room for evolving an effective and efficient texture descriptor. In this paper, we present a compact multi-pattern encoding (CMPE) descriptor for visual texture classification, which divides local neighborhood into local sign, local extremums, and center pixel. The contributions of this paper are highlighted as follows:

1. We introduce local extremum patterns, including maximum and minimum patterns, to represent local neighborhood intensity changes. Compared with traditional magnitude components, they have lower dimensionality and keep complete rotation invariance.

2. We develop a compact encoding scheme to jointly encode local maximum, minimum and center patterns into a three-bit binary code, named MMC pattern. No matter how many neighborhood points are sampled, its feature dimensionality always stays the same.

3. We propose a compact multi-pattern encoding descriptor by combining the traditional local sign pattern and MMC pattern to provide a highly complete texture representation.

4. Extensive Experiments on three popular texture databases validate that the compact multi-pattern encoding descriptor achieves the state-of-the-art classification performance.

The rest of the paper is organized as follows. In section 2, we briefly review the traditional LBP. Section 3, we present a compact multi-pattern encoding descriptor. Section 4 discusses the experimental results. Section 5 is the discussion. Finally, we conclude the paper in Section 6.