Skip to main content
SpringerLink
Log in

Image retrieval based on texture using latent space representation of discrete Fourier transformed maps

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Texture-based instance retrieval is typically performed on images that present a single texture pattern and is mainly applied to the retrieval of fabrics or textiles. In this work, we apply it to indoor scene images that typically present many different texture patterns, which constitutes a more challenging problem. Such retrieval systems, together with the retrieval of faces and objects, can be used as a valuable tool for evidence matching in crime scene investigation. Even though recent deep learning-based approaches have made significant improvement in many computer vision tasks, texture retrieval remains an open problem. In this work, we introduce a Fourier-based approach, in which spatial images and their discrete Fourier transform maps are combined to derive a novel texture representation. We further present a new and efficient texture-based image retrieval framework based on region proposal networks, convolutional autoencoders and transfer learning, in which we extract the features from the latent space layer of the encoder as texture descriptors. The experimental results on four datasets: TextileTube, Outex, USPtex and Stex, validated the effectiveness of our proposed method, yielding better results than the current state of the art.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. https://www.europol.europa.eu/stopchildabuse.

  2. https://ec.europa.eu/home-affairs/financing/fundings/projects/HOME_2010_ISE_AG_043_en.

  3. https://cordis.europa.eu/project/id/883341.

  4. http://gvis.unileon.es/dataset/textiltube/.

References

  1. Saritha RR, Paul V, Kumar PG (2019) Content based image retrieval using deep learning process. Cluster Comput 22(2):4187–4200

    Article  Google Scholar 

  2. Tzelepi M, Tefas A (2018) Deep convolutional learning for content based image retrieval. Neurocomputing 275:2467–2478

    Article  Google Scholar 

  3. Ahmed KT, Ummesafi S, Iqbal A (2019) Content based image retrieval using image features information fusion. Inf Fusion 51:76–99

    Article  Google Scholar 

  4. Alzu’bi A, Abuarqoub A (2020) Deep learning model with low-dimensional random projection for large-scale image search. Eng Sci Technol Int J 23:911–920

    Google Scholar 

  5. Forcen JI, Pagola M, Barrenechea E, Bustince H (2020) Co-occurrence of deep convolutional features for image search. Image Vis Comput 97:103909

    Article  Google Scholar 

  6. Keisler R, Skillman SW, Gonnabathula S, Poehnelt J, Rudelis X, Warren MS (2019) Visual search over billions of aerial and satellite images. Comput Vis Image Underst 187:102790102790

    Article  Google Scholar 

  7. Saikia S, Fidalgo E, Alegre E, Fernández-Robles L (2017) Object detection for crime scene evidence analysis using deep learning. In: International conference on image analysis and processing, Springer, pp 14–24

  8. Karie NM, Kebande VR, Venter H (2019) Diverging deep learning cognitive computing techniques into cyber forensics. Forensic Sci Int Synergy 1:61–67

    Article  Google Scholar 

  9. Mohammad RMA, Alqahtani M (2019) A comparison of machine learning techniques for file system forensics analysis. J Inf Secur Appl 46:53–61

    Google Scholar 

  10. Liu W, Wu CY (2019) Crime scene investigation image retrieval using a hierarchical approach and rank fusion. In: 2019 14th IEEE conference on industrial electronics and applications (ICIEA), IEEE, pp 1974–1978

  11. Liu Y, Hu D, Fan J, Wang F, Zhang D (2017) Multi-feature fusion for crime scene investigation image retrieval. In: 2017 international conference on digital image computing: techniques and applications (DICTA), IEEE, pp 1–7

  12. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: Towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, pp 91–99

  13. He K, Gkioxari G, Dollár P, Girshick R (2017) Mask r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp 2961–2969

  14. Nanni L, Brahnam S, Lumini A (2010) A local approach based on a local binary patterns variant texture descriptor for classifying pain states. Expert Syst Appl 37(12):7888–7894

    Article  Google Scholar 

  15. Nanni L, Lumini A, Brahnam S (2010) Local binary patterns variants as texture descriptors for medical image analysis. Artif intell Med 49(2):117–125

    Article  MATH  Google Scholar 

  16. Srinivasan G, Shobha G (2008) Statistical texture analysis. Proc World Acad Sci Eng Technol 36:1264–1269

    Google Scholar 

  17. Van de Wouwer G, Scheunders P, Van Dyck D (1999) Statistical texture characterization from discrete wavelet representations. IEEE Trans Image Process 8(4):592–598

    Article  Google Scholar 

  18. Wu Q, Wang J, Yang C, Cui G, Yang W (2016) Target recognition by texture segmentation algorithm. Expert Syst Appl 46:394–404

    Article  Google Scholar 

  19. Zheng L, Yang Y, Tian Q (2017) Sift meets cnn: a decade survey of instance retrieval. IEEE Trans Pattern Anal Mach Intell 40(5):1224–1244

    Article  Google Scholar 

  20. Singh B, Li H, Sharma A, Davis LS (2018) R-fcn-3000 at 30fps: Decoupling detection and classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1081–1090

  21. Tian Z, Shen C, Chen H, He T (2019) Fcos: Fully convolutional one-stage object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp 9627–9636

  22. Liu L, Ouyang W, Wang X, Fieguth P, Chen J, Liu X, Pietikäinen M (2020) Deep learning for generic object detection: a survey. Int J Comput Vis 128(2):261–318

    Article  Google Scholar 

  23. Wu X, Sahoo D, Hoi SC (2020) Recent advances in deep learning for object detection. Neurocomputing 396:39–64

    Article  Google Scholar 

  24. Ma W, Wu Y, Cen F, Wang G (2020) Mdfn: Multi-scale deep feature learning network for object detection. Pattern Recognit 100:107149

    Article  Google Scholar 

  25. Babenko A, Slesarev A, Chigorin A, Lempitsky V (2014) Neural codes for image retrieval. In: European conference on computer vision, Springer, pp 584–599

  26. Ng WW, Li J, Tian X, Wang H, Kwong S, Wallace J (2020) Multi-level supervised hashing with deep features for efficient image retrieval. Neurocomputing 399:171–182

    Article  Google Scholar 

  27. Wu Y, Wang S, Huang Q (2019) Multi-modal semantic autoencoder for cross-modal retrieval. Neurocomputing 331:165–175

    Article  Google Scholar 

  28. Daoud MI, Saleh A, Hababeh I, Alazrai R (2019) Content-based image retrieval for breast ultrasound images using convolutional autoencoders: A feasibility study. In: 2019 3rd International Conference on Bio-engineering for Smart Technologies (BioSMART), IEEE, pp 1–4

  29. Xu G, Fang W (2016) Shape retrieval using deep autoencoder learning representation. In: 2016 13th international computer conference on wavelet active media technology and information processing (ICCWAMTIP), IEEE, pp 227–230

  30. Ying G, Wei Q, Shen X, Han S (2008) A two-step phase-retrieval method in fourier-transform ghost imaging. Opt Commun 281(20):5130–5132

    Article  Google Scholar 

  31. Sokic E, Konjicija S (2016) Phase preserving fourier descriptor for shape-based image retrieval. Signal Process Image Commun 40:82–96

    Article  Google Scholar 

  32. Tsai DM, Tseng CF (1999) Surface roughness classification for castings. Pattern Recognit 32(3):389–405

    Article  Google Scholar 

  33. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, Ieee, pp 248–255

  34. Dai J, Li Y, He K, Sun J (2016) R-fcn: Object detection via region-based fully convolutional networks. In: Advances in neural information processing systems, pp 379–387

  35. Alzu’bi A, Amira A, Ramzan N (2015) Semantic content-based image retrieval: a comprehensive study. J Vis Commun Image Rep 32:20–54

    Article  Google Scholar 

  36. García-Olalla O, Alegre E, Fernández-Robles L, Fidalgo E, Saikia S (2018) Textile retrieval based on image content from cdc and webcam cameras in indoor environments. Sensors 18(5):1329

    Article  Google Scholar 

  37. Kwitt R, Uhl A (2008) Image similarity measurement by kullback-leibler divergences between complex wavelet subband statistics for texture retrieval. In: 2008 15th IEEE international conference on image processing, IEEE, pp 933–936

  38. Ouslimani F, Ouslimani A, Ameur Z (2019) Rotation-invariant features based on directional coding for texture classification. Neural Comput Appl 31(10):6393–6400

    Article  Google Scholar 

  39. Pham MT (2018) Efficient texture retrieval using multiscale local extrema descriptors and covariance embedding. In: Proceedings of the european conference on computer vision (ECCV)

  40. Tuncer T, Dogan S, Ertam F (2019) A novel neural network based image descriptor for texture classification. Phys A Stat Mech Appl 526:120955

    Article  MathSciNet  Google Scholar 

  41. Tuncer T, Dogan S, Ataman V (2019) A novel and accurate chess pattern for automated texture classification. Phys A Stat Mech Appl 536:122584

    Article  Google Scholar 

  42. King I, Lau TK (1996) A feature-based image retrieval database for the fashion, textile, and clothing industry in hong kong. In: Proceedings of international symposium multi-technology information processing, vol 96, pp 233–240

  43. D’Amato JP, Mercado M, Heiling A, Cifuentes V (2016) A proximal optimization method to the problem of nesting irregular pieces using parallel architectures. Rev Iberoam de Autom Inform Ind 13(2):220–227

    Article  Google Scholar 

  44. Wong C (2017) Applications of computer vision in fashion and textiles. Woodhead Publishing, Cambridge

    Google Scholar 

  45. Gordo A, Almazan J, Revaud J, Larlus D (2017) End-to-end learning of deep visual representations for image retrieval. Int J Comput Vis 124(2):237–254

    Article  MathSciNet  Google Scholar 

  46. Dos Santos JM, De Moura ES, Da Silva AS, da Silva Torres R (2017) Color and texture applied to a signature-based bag of visual words method for image retrieval. Multimed Tools Appl 76(15):16855–16872

    Article  Google Scholar 

  47. Mezaris V, Kompatsiaris I, Strintzis MG (2003) An ontology approach to object-based image retrieval. In: Proceedings 2003 international conference on image processing (Cat. No. 03CH37429), IEEE, vol 2, pp II–511

  48. Ren J, Shen Y, Guo L (2003) A novel image retrieval based on representative colors. In: Proceedings of the Image and Vision Computing, NZ, Citeseer

  49. Song J, Gao L, Liu L, Zhu X, Sebe N (2018) Quantization-based hashing: a general framework for scalable image and video retrieval. Pattern Recognit 75:175–187

    Article  Google Scholar 

  50. Maillot N, Thonnat M, Boucher A (2004) Towards ontology-based cognitive vision. Mach Vis Appl 16(1):33–40

    Article  MATH  Google Scholar 

  51. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  52. Bay H, Ess A, Tuytelaars T, Van Gool L (2008) Speeded-up robust features (surf). Comput Vis Image Underst 110(3):346–359

    Article  Google Scholar 

  53. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), IEEE, vol 1, pp 886–893

  54. Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987

    Article  MATH  Google Scholar 

  55. Verma M, Raman B (2018) Local neighborhood difference pattern: a new feature descriptor for natural and texture image retrieval. Multimed Tools Appl 77(10):11843–11866

    Article  Google Scholar 

  56. Nan B, Xu Y, Mu Z, Chen L (2015) Content-based image retrieval using local texture-based color histogram. In: 2015 IEEE 2nd international conference on cybernetics (CYBCONF), IEEE, pp 399–405

  57. Singh C, Walia E, Kaur KP (2018) Color texture description with novel local binary patterns for effective image retrieval. Pattern Recognit 76:50–68

    Article  Google Scholar 

  58. Pavithra L, Sharmila TS (2018) An efficient framework for image retrieval using color, texture and edge features. Comput Electrl Eng 70:580–593

    Article  Google Scholar 

  59. Yang C, Yu Q (2019) Multiscale fourier descriptor based on triangular features for shape retrieval. Signal Process Image Commun 71:110–119

    Article  Google Scholar 

  60. Liu W, Wang Z, Liu X, Zeng N, Liu Y, Alsaadi FE (2017) A survey of deep neural network architectures and their applications. Neurocomputing 234:11–26

    Article  Google Scholar 

  61. Popa CA, Cernăzanu-Glăvan C (2018) Fourier transform-based image classification using complex-valued convolutional neural networks. In: International symposium on neural networks, Springer, pp 300–309

  62. Chitsaz K, Hajabdollahi M, Karimi N, Samavi S, Shirani S (2020) Acceleration of convolutional neural network using fft-based split convolutions. arXiv preprint arXiv:200312621

  63. Saikia S, Fidalgo E, Alegre E, Fernández-Robles L (2017) Query based object retrieval using neural codes. International Joint Conference SOCO’17-CISIS’17-ICEUTE’17 León, Spain, September 6–8, 2017. Springer, Proceeding, pp 513–523

  64. Salvador A, Giró-i Nieto X, Marqués F, Satoh S (2016) Faster r-cnn features for instance search. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 9–16

  65. Cimpoi M, Maji S, Kokkinos I, Mohamed S, Vedaldi A (2014) Describing textures in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3606–3613

  66. Cimpoi M, Maji S, Vedaldi A (2015) Deep filter banks for texture recognition and segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3828–3836

  67. Cimpoi M, Maji S, Kokkinos I, Vedaldi A (2016) Deep filter banks for texture recognition, description, and segmentation. Int J Comput Vis 118(1):65–94

    Article  MathSciNet  Google Scholar 

  68. Yikun Y, Shengjie J, Jinrong H, Bisheng X, Jiabo L, Ru X (2020) Image retrieval via learning content-based deep quality model towards big data. Future Gener Comput Syst 112:243–249

    Article  Google Scholar 

  69. Redmon J, Farhadi A (2018) Yolov3: An incremental improvement. arXiv preprint arXiv:180402767

  70. Abdelmounaime S, Dong-Chen H (2013) New brodatz-based image databases for grayscale color and multiband texture analysis. ISRN Machine Vision 2013

  71. Casanova D, Florindo JB, Falvo M, Bruno OM (2016) Texture analysis using fractal descriptors estimated by the mutual interference of color channels. Inf Sci 346:58–72

    Article  Google Scholar 

  72. Kwitt R, Meerwald P (2018) Salzburg texture image database (stex)

  73. Guo JM, Prasetyo H, Wang NJ (2015) Effective image retrieval system using dot-diffused block truncation coding features. IEEE Trans Multimed 17(9):1576–1590

    Article  Google Scholar 

  74. Napoletano P (2017) Hand-crafted vs learned descriptors for color texture classification. In: International workshop on computational color imaging, Springer, pp 259–271

Download references

Acknowledgements

This work has been supported by the grant Junta de Castilla y Leon (EDU/529/2017) and the framework agreement between the University of Leon and INCIBE (Spanish National Cybersecurity Institute) under Addendum 01. We gratefully acknowledge the support of Nvidia Corporation for their kind donation of GPUs (GeForce GTX Titan X and K-40).

Author information

Authors and Affiliations

  1. Department of Electrical, Systems and Automation Engineering, Universidad de León, León, Spain

    Surajit Saikia, Enrique Alegre & Eduardo Fidalgo

  2. Researcher at INCIBE (Spanish National Cybersecurity Institute), León, Spain

    Surajit Saikia, Laura Fernández-Robles, Enrique Alegre & Eduardo Fidalgo

  3. Department of Mechanical, Informatics and Aerospace Engineering, Universidad de León, León, Spain

    Laura Fernández-Robles

Authors
  1. Surajit Saikia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Laura Fernández-Robles
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Enrique Alegre
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eduardo Fidalgo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Surajit Saikia.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Saikia, S., Fernández-Robles, L., Alegre, E. et al. Image retrieval based on texture using latent space representation of discrete Fourier transformed maps. Neural Comput & Applic 33, 13301–13316 (2021). https://doi.org/10.1007/s00521-021-05955-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-05955-2

Keywords

Search

Navigation