Skip to main content
SpringerLink
Log in

Exploiting deep textures for image retrieval

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Deep features and texture features each have advantages in image representation. However, exploiting deep textures for image retrieval is challenging because it is difficult to enhance the compatibility of texture features and deep features. To address this problem, we propose a novel image-retrieval method named the deep texture feature histogram (DTFH). The main highlights are: (1) We propose a novel method for identifying effective, limited-effectiveness, or non-valid feature maps via ranking based on Haralick’s statistics, which can help understand image content and identify objects, as these statistics have clear physical significance. (2) We use Gabor filtering to mimic the human orientation-selection mechanism, which allows deep texture features to contain a good representation of orientation, thereby enhancing discriminative power. (3) We combine the advantages of classical texture features and deep features to provide a compact representation. This provides a new, yet simple way to exploit deep features via the use of traditional classical texture features. Comparative experiments demonstrate that deep texture features provide highly competitive performance in image retrieval in terms of mean average precision (mAP), and provide new insights into the exploitation of traditional texture features and deep features.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Tyagi V (2017) Content-based image retrieval: ideas, influences, and current trends. Springer, Singapore

    Book  MATH  Google Scholar 

  2. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems, pp. 1097–1105

  3. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. https://arXiv.org/1409.1556

  4. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778

  5. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L (2018) MobileNetV2: inverted residuals and linear bottlenecks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 4510–4520

  6. Tan M, Le QV (2019) EfficientNet: rethinking model scaling for convolutional neural networks. ArXiv Preprint https://arXiv.org/abs/1905.1194

  7. Liu G-H, Yang J-Y (2008) Image retrieval based on the texton co-occurrence matrix. Pattern Recogn 41(12):3521–3527

    Article  MATH  Google Scholar 

  8. Liu G-H, Yang J-Y, Li ZY (2015) Content-based image retrieval using computational visual attention model. Pattern Recogn 48(8):2554–2566

    Article  Google Scholar 

  9. Haralick RM, Shanmugam K, Dinstein I (1973) Textural feature for image classification. IEEE Trans Syst Man Cybern SMC 3(6):610–621

    Article  Google Scholar 

  10. Manjunath BS, Ma WY (1996) Texture features for browsing and retrieval of image data. IEEE Trans Pattern Anal Mach Intell 18(8):837–842

    Article  Google Scholar 

  11. Manjunath BS, Salembier P, Sikora T (2002) Introduction to MPEG-7: multimedia content description interface. Wiley

    Google Scholar 

  12. Liu G-H, Yang J-Y (2019) Exploiting color volume and color difference for salient region detection. IEEE Trans Image Process 28(1):6–16

    Article  MATH  Google Scholar 

  13. Liu G-H, Yang J-Y (2021) Deep-seated features histogram: a novel image retrieval method. Pattern Recogn 116:107926

    Article  Google Scholar 

  14. Liu G-H, Yang J-Y (2013) Content-based image retrieval using color deference histogram. Pattern Recogn 46(1):188–198

    Article  Google Scholar 

  15. Hua J-Z, Liu G-H, Song S-X (2019) Content-based image retrieval using color volume histograms. Int J Pattern Recognit Artif Intell 33(9):1940010

    Article  Google Scholar 

  16. Liu G-H, Wei Z (2020) Image retrieval using the fused perceptual color histogram. Comput Intell Neurosci 2020:10 (Article ID 8876480)

    Article  Google Scholar 

  17. Cross G, Jain A (1983) Markov random field texture models. IEEE Trans Pattern Anal Mach Intell 5(1):25–39

    Article  Google Scholar 

  18. Ojala T, Pietikanen M, Maenpaa T (2002) Multi-resolution grayscale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987

    Article  Google Scholar 

  19. Thompson EM, Biasotti S (2018) Description and retrieval of geometric patterns on surface meshes using an edge-based LBP approach. Pattern Recogn 82:1–15

    Article  Google Scholar 

  20. Dubey SR, Singh SK, Singh RK (2016) Multichannel decoded local binary patterns for content-based image retrieval. IEEE Trans Image Process 25(9):4018–4032

    Article  MATH  Google Scholar 

  21. Singh C, Walia E, Kaur KP (2017) Color texture description with novel local binary patterns for effective image retrieval. Pattern Recogn 76:50–68

    Article  Google Scholar 

  22. Murala S, Maheshwari RP, Balasubramanian R (2012) Local tetra patterns: a new feature descriptor for content-based image retrieval. IEEE Trans Image Process 21(5):2874–2886

    Article  MATH  Google Scholar 

  23. Chu K, Liu G-H (2020) Image retrieval based on a multi-integration features model. Math Probl Eng 2020:1461459

    Article  Google Scholar 

  24. Liu G-H, Li Z-Y, Zhang L, Xu Y (2011) Image retrieval based on micro-structure descriptor. Pattern Recogn 44(9):2123–2133

    Article  Google Scholar 

  25. Wei Z, Liu G-H (2020) Image retrieval using the intensity variation descriptor. Math Probl Eng 2020:12 (Article ID 6283987)

    Article  Google Scholar 

  26. Liu G-H, Zhang L et al (2010) Image retrieval based on multi-texton histogram. Pattern Recogn 43(7):2380–2389

    Article  MATH  Google Scholar 

  27. Clement M, Kurtz C, Wendling L (2018) Learning spatial relations and shapes for structural object description and scene recognition. Pattern Recogn 84:197–210

    Article  Google Scholar 

  28. Hong B, Soatto S (2015) Shape matching using multiscale integral invariants. IEEE Trans Pattern Anal Mach Intell 37(1):151–160

    Article  Google Scholar 

  29. Žunić J, Rosin PL, Ilić V (2018) Disconnectedness: a new moment invariant for multi-component shapes. Pattern Recogn 78:91–102

    Article  Google Scholar 

  30. Malu G, Elizabeth S, Koshy SM (2018) Circular mesh-based shape and margin descriptor for object detection. Pattern Recogn 84:97–111

    Article  Google Scholar 

  31. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vision 60(2):91–110

    Article  Google Scholar 

  32. Bay H, Tuytelaars T, Gool LV (2006) SURF: speeded up robust features. Eur Conf Comput Vis 1:404–417

    Google Scholar 

  33. Mikolajczyk K, Tuytelaars T, Schmid C et al (2005) A comparison of affine region detectors. Int J Comput Vis 65(1–2):43–72

    Article  Google Scholar 

  34. Mikolajczyk K, Schmid C (2005) A performance evaluation of local descriptors. IEEE Trans Pattern Anal Mach Intell 27(10):1615–1630

    Article  Google Scholar 

  35. Sivic J, Zisserman A (2009) Efficient visual search of videos cast as text retrieval. IEEE Trans Pattern Anal Mach Intell 31(4):591–606

    Article  Google Scholar 

  36. Perronnin F, Liu Y, Sánchez J, Poirier H (2010) Large-scale image retrieval with compressed Fisher vectors. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, pp. 3384–3391

  37. Jégou H, Perronnin F, Douze M, Sánchez J, Pérez P, Schmid C (2012) Aggregating local image descriptors into compact codes. IEEE Trans Pattern Anal Mach Intell 34(9):1704–1716

    Article  Google Scholar 

  38. Husain SS, Bober M (2017) Improving large-scale image retrieval through robust aggregation of local descriptors. IEEE Trans Pattern Anal Mach Intell 39(9):1783–1796

    Article  Google Scholar 

  39. Jégou H, Zisserman A (2014) Triangulation embedding and democratic aggregation for image search. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, pp. 3310–3317

  40. Babenko A, Slesarev A, Chigorin A, Lempitsky V (2014) Neural codes for image retrieval. In: Proceedings of the European Conference on Computer Vision, pp 584–599

  41. Ng JY, Yang F, Davis LS (2015) Exploiting local features from deep networks for image retrieval. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Boston, MA, pp. 53–61

  42. Tolias G, Sicre R, Jégou H (2016) Particular object retrieval with integral max-pooling of CNN activations. In: Proceedings of International Conference on Learning Representations

  43. Arandjelović R, Gronat P, Torii A, Pajdla T, Sivic J (2018) NetVLAD: CNN architecture for weakly supervised place recognition. IEEE Trans Pattern Anal Mach Intell 40(6):1437–1451

    Article  Google Scholar 

  44. Kalantidis Y, Mellina C, Osindero S (2016) Cross-dimensional weighting for aggregated deep convolutional features. In: Proceedings of the European Conference on Computer Vision, pp 685–701

  45. Radenović F, Tolias G, Chum O (2019) Fine-tuning CNN image retrieval with no human annotation. IEEE Trans Pattern Anal Mach Intell 41(7):1655–1668

    Article  Google Scholar 

  46. Yandex AB, Lempitsky V (2015) Aggregating deep convolutional features for image retrieval. In: 2015 IEEE International Conference on Computer Vision, pp 1269–1277

  47. Mohedano E, McGuinness K, O’Connor EN et al. (2016) Bags of local convolutional features for scalable instance search. In: Proceedings of the ACM on International Conference on Multimedia Retrieval, pp. 327–331

  48. Gordo A, Almazán J, Revaud J, Larlus D (2016) Deep image retrieval: learning global representations for image search. In: Proceedings of the European Conference on Computer Vision, pp 241–257

  49. Gordo A, Almazán J, Revaud J, Larlus D (2017) End-to-end learning of deep visual representations for image retrieval. Int J Comput Vis 124(2):237–254

    Article  Google Scholar 

  50. Revaud J, Almazan J, Rezende R, Souza CD (2019) Learning with average precision: training image retrieval with a listwise loss. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), pp 5106–5115

  51. Husain SS, Ong EJ, Bobber M (2021) ACTNET: end-to-end learning of feature activations and multi-stream aggregation for effective instance image retrieval. Int J Comput Vis 129:1432–1450

    Article  Google Scholar 

  52. Zheng L, Yang Y, Tian Q (2018) SIFT Meets CNN: a decade survey of instance retrieval. IEEE Trans Pattern Anal Mach Intell 40(5):1224–1244

    Article  Google Scholar 

  53. Nister D, Stewenius H (2006) Scalable recognition with a vocabulary tree. IEEE Comput Soc Conf Comput Vis Pattern Recogn 2006:2161–2168

    Google Scholar 

  54. Philbin J, Chum O, Isard M, Sivic J, Zisserman A (2007) Object retrieval with large vocabularies and fast spatial matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

  55. Philbin J, Chum O, Isard M, Sivic J, Zisserman A (2008) Lost in quantization: improving particular object retrieval in large scale image databases. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, pp. 1–8

  56. Jegou H, Douze M, Schmid C (2008) Hamming embedding and weak geometry consistency for large scale image search. In: Proceedings of the 10th European conference on Computer vision, October 2008

  57. Jégou H, Chum O (2012) Negative evidence and co-occurences in image retrieval: the benefit of PCA and whitening. In: Proc. Eur. Conf. Comput. Vis., pp. 774–787

Download references

Funding

This study is supported by National Natural Science Foundation of China (grant no. 61866005 and 62266008), the Foundation of Guangxi Normal University (grant no. 2021JC007), and the Foundation of Development Research Center of Guangxi in Humanities and Social Sciences (grant no. ZXZJ202201).

Author information

Authors and Affiliations

  1. College of Computer Science and Engineering, Guangxi Normal University, Guilin, 541004, China

    Guang-Hai Liu

  2. School of Computer Science and Technology, Nanjing University of Science and Technology, Nanjing, 210094, China

    Jing-Yu Yang

Authors
  1. Guang-Hai Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jing-Yu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guang-Hai Liu.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, GH., Yang, JY. Exploiting deep textures for image retrieval. Int. J. Mach. Learn. & Cyber. 14, 483–494 (2023). https://doi.org/10.1007/s13042-022-01645-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-022-01645-0

Keywords

Search

Navigation