Abstract
In local pattern based feature extraction, usually the raw spatial image provides limited information about the relationship between the pixels in a local neighborhood. A few recent methods address this issue by first filtering the images with bag of filters and then calculating local pattern over each filtered images. It is observed that the filtered images complement the discriminativeness of local pattern based features which enhances retrieval efficiency. Motivated by these approaches, a new approach based on multiple filters and decoded sparse local binary pattern (MF-DLBP) is proposed in this paper, wherein we first filter the raw spatial image with multiple filters to extract the low frequency and high frequency information. However, unlike previous approaches, we extract features from low pass and high pass filtered images adopting separate strategies, since characteristically they contain contrasting information. From each gradient filtered images, we compute sparse LBPs using LBP4hv (considering only horizontal and vertical neighbors) and LBP4d (considering only diagonal neighbors) techniques. To enhance the discriminativeness of the descriptor, two decoders are also used which compute the inter frequency relationship between the sparse local binary pattern maps of high pass filtered images only. The low pass filtered image is encoded with sign LBP (\(LBP\_S\)) and magnitude LBP (\(LBP\_M\)) to extract sign as well as magnitude information present in this image. The proposed approach is low dimensional and it shows highly competitive retrieval performance when tested on three benchmark texture databases which are Kylberg, Brodatz and STex.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
Texture analysis and retrieval has immense value in computer vision and pattern recognition and is also very useful in industrial identification applications. Both texture analysis and retrieval have gained ample attention among the researchers for past few decades. The vast amount of works in this field can be categorized into several categories such as statistical, structural, model based, learning based approaches etc. [10]. Recently introduced deep leaning based approaches are also shown to be very efficient for retrieval purpose [11, 16]. However, the deep learning networks need to be pre-trained with large volume of training data. Besides these methods are often time consuming and their memory requirements are very high. Recently, local pattern based texture classification is also gaining popularity among the researchers because of it’s easy to understand and adaptive feature extraction architecture. Local binary pattern (LBP) is the pioneering work in this field [15]. To enhance the efficacy of LBP in retrieval, a number of new feature descriptors have also been proposed, which extracts the local texture information in more efficient manner [4, 7, 12, 14, 17, 20]. High retrieval efficiencies have been reported for most of these approaches; however the feature dimension of most of these approaches are also very large. Some of these methods are suitable for only a particular kind of database. These limitations motivated us to develop a new feature descriptor for texture images, which has comparatively low feature dimension yet highly effective for retrieval purpose. Since, LBP encodes only the sign of the differences between the center pixel and the neighboring pixels and doesn’t consider the magnitude of difference values, in [7], a completed modeling of LBP (CLBP) was proposed, considering both sign and magnitude information. The effect of adjacent neighbors in a local neighborhood on image texture is analyzed in [20]. Both these methods are shown to improve the retrieval performance of LBP in various image databases. The fusion of co-occurrence information with local pattern to enhance the effectiveness of the features is investigated in [8, 9, 13, 19]. These approaches usually show state of art performance for retrieval of texture, face and bio medical images. However, one drawback of these approaches is their higher feature dimension, which may slow down the retrieval process [8, 13, 19]. In [4, 5], feature extraction from the filtered images using local pattern is explored for retrieval. Both these approaches show that the discriminative capability of the local features computed from edge detected filtered images are superior to those computed from the original raw images.
The methods proposed in [4, 5, 7] inspired us to propose a new feature descriptor for retrieval of texture images. We first filter the raw images with one low pass and multiple high pass filters. Then unlike in [4, 5], features are extracted from the low and high pass filtered images using different techniques. Since low pass filtered image contains more information than a high pass filtered image, we propose to encode it using sign LBP and magnitude LBP. In order to limit the feature dimension, we compute uniform histograms from the encoded low pass maps. However for encoding the multiple high pass filtered images we use sparse LBP in order to shorten the feature dimension. To further improve the discrimination capability of the descriptor the mutual inter-frequency information among the high pass LBP maps are computed using two decoders.
The paper is organized as follows: In Sect. 1, a brief review of some of the works related to texture analysis and retrieval is presented. Section 2 presents the proposed method. In Sect. 3, the experimental results and relevant discussions are provided. The paper is summarized and concluded in Sect. 4.
2 Proposed Method
The proposed method is depicted in a schematic block diagram in Fig. 1. The feature extraction procedure of the proposed descriptor can be summarized into following steps:
2.1 Filtering of the Image with Multiple Filters
In this step, the image is filtered with an averaging filter (\(F_A\)) and three high pass filters which are Sobel horizontal (\(F_H\)), Sobel vertical (\(F_V\)) and Sobel diagonal (\(F_D\)) edge filters [6]. All the four filtering masks considered here are shown in Fig. 2. The filtered images are obtained from these masks using following equation-
where, G is the input image, \(F_i\) is the filtering mask with \(i\in \left[ A, H, V, D\right] \), \(I_i\) is the filtered image corresponding to \(F_i\) and \(*\) denotes the convolution operation on image G by \(F_i\).
2.2 Encoding of the Low Pass Image
We employ sign LBP (\(LBP\_S\)) and magnitude LBP (\(LBP\_M\)) approaches to extract the features from the low pass filtered image \(I_A\) using following equations:
and
where, \(N_1\) is the number of neighbors in the local neighborhood of radius R (Here R \(=\) 1 and \(N_1=\) 8) and ‘Th’ is an empirical threshold which is taken here as 25, \(I_A^i\) and \(I_A^c\) are the neighboring pixels and center pixel respectively.
Uniform histograms [18] are computed from both these pattern maps and are concatenated to construct the feature vector \(F_1\).
The dimension of \(F_1\) alone is \(2 \times 59 = 118\) features.
2.3 Encoding of the High Pass Filtered Images
The high pass filtered images which contain fine details are encoded with sparse LBP. In computation of sparse LBP, we first consider only four diagonal neighbors among the eight neighbors present in a local neighborhood of radius 1 as shown in Fig. 3(c). Then LBP map (LBP4d) is calculated considering only these four diagonal neighbors along with the center pixel as shown in Fig. 3(e). Similarly considering only the horizontal and vertical neighbors and the center pixel (Fig. 3(d)), the second LBP pattern map (LBP4hv) is formed as shown in Fig. 3(f).
LBP4d and LBP4hv are calculated from \(I_H, I_V\) and \(I_D\) images as shown in Fig. 3. After computing LBP4d and LBP4hv from \(I_H, I_V\) and \(I_D\) images, we get six binary pattern maps which are HLBP4d, HLBP4hv (corresponding to \(I_H\)), VLBP4d, VLBP4hv (corresponding to \(I_V\)), DLBP4d and DLBP4hv (corresponding to \(I_D\)) as shown in Fig. 1. The advantage of this sparse approach is that it minimizes the feature dimension to a great extent by providing pattern values ranging between 0 to 15 only.
2.4 Computation of Inter Frequency Relationship Among the Encoded Pattern Maps of High Pass Images
To enhance the discrimination capability of the features, the inter frequency relationship among the LBP pattern maps of the high pass filtered images are computed with the help of two 3:8 decoders. Each of these decoders take three 4 bit LBP map as input from horizontal, vertical and diagonal filtered images respectively and then decode them into eight 4 bit LBP maps at the output. Therefore, HLBP4d, VLBP4d and DLBP4d maps are fed as input into the first decoder, while HLBP4hv, VLBP4hv and DLBP4hv are fed as input into the second decoder. The binary pattern outputs of both the decoders are multiplied by their corresponding weights and is summed up to a pattern value (the value ranges between 0 to 15). Therefore, a total of sixteen decoded maps \(MF-DLBP4d^t\) and \(MF-DLBP4hv^t\) (\(t\,\epsilon \,\left[ 0,7\right] \)) respectively are obtained. The working of the decoder to obtain \(MF-DLBP4d^t\) maps is illustrated using an example in Fig. 4. The truth table for the decoders are given in Table 1.
The feature vector (\(F_2\)) is constructed by computing histograms of all \(MF-DLBP4\delta ^t\) (\(\delta =d\) and hv] and \(t\in \left[ 0,7\right] \)) maps and concatenating all of them as
The dimension of \(F_2\) alone is \(16 \times 16 = 256\) features.
2.5 Final Feature Vector Formation
The final feature vector (FV) is constructed by concatenating the features extracted from low pass filter image i.e \(F_1\) with the features of the high pass filtered images i.e \(F_2\). Therefore,
The total feature vector length of proposed descriptor is \(118 + 256 = 374\) features.
3 Results and Discussions
In order to evaluate retrieval performance of the proposed method, experiments are conducted on three benchmark texture image databases namely Kylberg [2], Brodatz [3] and STex [1]. The summary of these databases are presented in Table 2. The retrieval performances are evaluated using two standard parameters namely Average Retrieval Precision (ARP) and Average Retrieval Recall (ARR) [19]. Higher values of them indicate better retrieval performance. In order to compute them, every image in the database is considered as query image once for each database and against each query, ‘n’ number of closely matched images are retrieved. The term ‘n’ is the number of top matches considered. Similarity between query image ‘Q’ and database images are computed using ‘\(d_1\)’ distance measure [20] for the proposed method. The retrieval performance of the proposed method is compared with six well known local pattern based methods: LBP (Dim: 256) [15], CSLBCoP (Dim: 1024) [19], BOF-LBP (Dim: 1280) [5], LTriDP (Dim: 768) [20], CoALTP (Dim: 2048) [13] and FDLBP (Dim: 4096) [4]. Except BOF-LBP which we have implemented ourselves, the retrieval results of all other techniques were obtained using matlab codes provided by their original authors.
The first experiment is conducted on Kylberg database [2]. To evaluate the retrieval performance, the images are retrieved in groups of 2, 4, 6..., 32 images and ARP and ARR values are calculated for each group of top matches. The plots for ARP and ARR are presented in Fig. 5(a–b). From the plots, it is observed that in most of the top matches, the proposed approach out performs all the other approaches. However in higher top matches (top matches of 28 or higher) however, CoALTP and CSLBCoP perform slightly better than the proposed approach. However, it should be noted that CoALTP and CSLBCoP have feature dimension much larger than the proposed approach. Overall, in this database, for 32 top matches, the proposed approach shows percentage improvement of 6.71% over LBP, −0.22% over CSLBCoP, 3.42% over BOF-LBP, 2.11% over LTriDP, −0.88% over CoALTP and 5.76% over FDLBP both in terms of ARP and ARR.
We conducted the second experiment on Brodatz database, [3]. In the experiment, the images are retrieved considering 5, 10, 15, 20 and 25 number of top matches. The ARP and ARR plots are shown in Fig. 5(c-d), which suggest that in this database, the proposed method consistently outperforms all the other methods both in terms of ARP and ARR. For 25 top matches, the proposed approach improves performance of LBP by 11.92%, of CSLBCoP by 4.40%, of BOF-LBP by 2.05%, of LTriDP by 2.57%, of CoALTP by 3.08% and of FDLBP by 6.82% both in terms of ARP and ARR in Brodatz database.
In our third experiment, Salzburg Texture Image Database (STex) is considered [1]. Since the images of this database are color images, as part of pre-processing, we first converted the images into gray scale images, before applying the feature descriptors. To evaluate the retrieval performance, we retrieved the images in groups of 2, 4, 6,...16 top matches. We present the ARP and ARR plots in Fig. 5(e–f) and from the plots it can be seen that as compared to all the techniques, the proposed approach achieves the best performance in STex database, at all the top matches. The proposed approach has shown percentage improvement of 26.76% over LBP, 6.97% over CSLBCoP, 10.30% over BOF-LBP, 6.67% over LTriDP, 6.01% over CoALTP and 8.87% over FDLBP both in terms of ARP and ARR, for 16 number of top matches.
The retrieval performance of LBP, CSLBCoP, BOF-LBP, LTriDP, CoALTP, FDLBP and MF-DLBP techniques for maximum number of top matches and their corresponding feature vector dimensions are presented in Table 3.
Table 3 shows only LBP has less feature dimension than the proposed method. The feature vector of the proposed method is 1.46 times higher than that of LBP, whereas the feature dimensions of CSLBCoP, BOF-LBP, LTriDP, CoALTP and FDLBP are 2.73, 3.42, 2.05, 5.47, 10.95 times higher than the proposed method. However, the proposed method still outperforms all the other methods in most of the cases.
4 Conclusion
In this article, a new feature descriptor MF-DLBP is proposed for efficient texture retrieval. The descriptor first filter the image using one low pass and multiple high pass filters and then to extract the features, the low and high pass images are encoded using separate techniques. While the low pass image is encoded with \(LBP\_S\) and \(LBP\_M\) techniques to extract both sign and magnitude information of the image texture, the high pass images are encoded using a sparse LBP technique. The features are extracted from the encoded low pass images by calculating uniform histograms from them. On the other hand, unlike FDLBP, two decoders are used to compute the inter frequency relation among the encoded LBP maps of the high pass images only, which further enhances the discrimination capability of the extracted features. When conducted experiments on three bench mark texture databases, it is observed that the proposed approach with significantly smaller feature dimensions shows state of the art retrieval performance.
References
Salzburg texture image database (STex). www.wavelab.at/sources/STex/. Accessed February 2019
Kylberg texture dataset. http://www.cb.uu.se/~gustaf/texture/. Accessed November 2018
Brodatz texture database. https://multibandtexture.recherche.usherbrooke.ca/original_brodatz.html. Accessed October 2018
Dubey, S.R.: Face retrieval using frequency decoded local descriptor. Multimed. Tools Appl., 1–21 (2018)
Dubey, S.R., Singh, S.K., Singh, R.K.: Boosting local binary pattern with bag-of-filters for content based image retrieval. In: 2015 IEEE UP Section Conference on Electrical Computer and Electronics (UPCON), pp. 1–6. IEEE (2015)
Gonzalez, R.C., Woods, R.E.: Digital Image Processing. Pearson Education, London (2002)
Guo, Z., Zhang, L., Zhang, D.: A completed modeling of local binary pattern operator for texture classification. IEEE Trans. Image Process. 19(6), 1657–1663 (2010)
Hatibaruah, R., Nath, V., Saikia, K., Hazarika, D.: Elliptical local binary co-occurrence pattern for face image retrieval. J. Stat. Manag. Syst. 22(2), 223–236 (2019)
Hatibaruah, R., Nath, V.K., Hazarika, D.: An effective texture descriptor for retrieval of biomedical and face images based on co-occurrence of similar center-symmetric local binary edges. Int. J. Comput. Appl., 1–12 (2019)
Humeau-Heurtier, A.: Texture feature extraction methods: a survey. IEEE Access 7, 8975–9000 (2019)
Li, C., Huang, Y.: Deep decomposition of circularly symmetric Gabor wavelet for rotation-invariant texture image classification. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 2702–2706. IEEE (2017)
Murala, S., Maheshwari, R., Balasubramanian, R.: Local tetra patterns: a new feature descriptor for content-based image retrieval. IEEE Trans. Image Process. 21(5), 2874–2886 (2012)
Naghashi, V.: Co-occurrence of adjacent sparse local ternary patterns: a feature descriptor for texture and face image retrieval. Optik 157, 877–889 (2018)
Nath, V.K., Hatibaruah, R., Hazarika, D.: An efficient multiscale wavelet local binary pattern for biomedical image retrieval. In: Mandal, J.K., Saha, G., Kandar, D., Maji, A.K. (eds.) Proceedings of the International Conference on Computing and Communication Systems. LNNS, vol. 24, pp. 489–497. Springer, Singapore (2018). https://doi.org/10.1007/978-981-10-6890-4_47
Ojala, T., Pietikäinen, M., Harwood, D.: A comparative study of texture measures with classification based on featured distributions. Pattern Recogn. 29(1), 51–59 (1996)
Özuysal, M.: Ground texture classification with deep learning. In: 2018 26th Signal Processing and Communications Applications Conference (SIU), pp. 1–4. IEEE (2018)
Tan, X., Triggs, W.: Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans. Image Process. 19(6), 1635–1650 (2010)
Topi, M., Timo, O., Matti, P., Maricor, S.: Robust texture classification by subsets of local binary patterns. In: Proceedings 15th International Conference on Pattern Recognition. ICPR-2000, vol. 3, pp. 935–938. IEEE (2000)
Verma, M., Raman, B.: Center symmetric local binary co-occurrence pattern for texture, face and bio-medical image retrieval. J. Vis. Commun. Image Represent. 32, 224–236 (2015)
Verma, M., Raman, B.: Local tri-directional patterns: a new texture feature descriptor for image retrieval. Digit. Sig. Proc. 51, 62–72 (2016)
Acknowledgments
This work was supported by Digital India Corporation (formerly Media Lab Asia), Ministry of Electronics and Information Technology, Govt. of India, through Visvesvaraya Ph.D scheme.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Hatibaruah, R., Nath, V.K., Hazarika, D. (2019). Texture Image Retrieval Using Multiple Filters and Decoded Sparse Local Binary Pattern. In: Deka, B., Maji, P., Mitra, S., Bhattacharyya, D., Bora, P., Pal, S. (eds) Pattern Recognition and Machine Intelligence. PReMI 2019. Lecture Notes in Computer Science(), vol 11941. Springer, Cham. https://doi.org/10.1007/978-3-030-34869-4_59
Download citation
DOI: https://doi.org/10.1007/978-3-030-34869-4_59
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-34868-7
Online ISBN: 978-3-030-34869-4
eBook Packages: Computer ScienceComputer Science (R0)