Skip to main content
SpringerLink
Log in

Vector space model adaptation and pseudo relevance feedback for content-based image retrieval

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Image retrieval is an important problem for researchers in computer vision and content-based image retrieval (CBIR) fields. Over the last decades, many image retrieval systems were based on image representation as a set of extracted low-level features such as color, texture and shape. Then, systems calculate similarity metrics between features in order to find similar images to a query image. The disadvantage of this approach is that images visually and semantically different may be similar in the low level feature space. So, it is necessary to develop tools to optimize retrieval of information. Integration of vector space models is one solution to improve the performance of image retrieval. In this paper, we present an efficient and effective retrieval framework which includes a vectorization technique combined with a pseudo relevance model. The idea is to transform any similarity matching model (between images) to a vector space model providing a score. A study on several methodologies to obtain the vectorization is presented. Some experiments have been undertaken on Wang, Oxford5k and Inria Holidays datasets to show the performance of our proposed framework.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. http://wang.ist.psu.edu/docs/related/

  2. http://www.robots.ox.ac.uk/vgg/data/oxbuildings/

  3. https://lear.inrialpes.fr/jegou/data.php

References

  1. Aidos H, Duarte JMM, Fred ALN (2014) Identifying regions of interest for discriminating Alzheimer’s disease from mild cognitive impairment. IEEE International Conference on Image Processing (ICIP), pp 21–25

  2. Angelova A, Zhu S (2013) Efficient object detection and segmentation for fine-grained recognition. IEEE Conference on Computer Vision and Pattern Recognition, pp 811–818

  3. Anh ND, Bao PT, Nam BN, Hoang NH (2010) A New CBIR System Using SIFT Combined with Neural Network and Graph-Based Segmentation. Intelligent Information and Database Systems, Second International Conference, ACIIDS, pp 294–301

  4. Alpkocak A, Kilinc D, Expansion TB (2010) Re-ranking approaches for multimodal image retrieval using text-based methods ImageCLEF, Experimental Evaluation in Visual Information Retrieval, pp 261–275

  5. Arandjelovic R, Gronat P, Torii A, Pajdla T, Sivic J (2015) NetVLAD: CNN architecture for weakly supervised place recognition

  6. Arandjelovic R, Zisserman A (2012) Three things everyone should know to improve object retrieval. IEEE Conference on Computer Vision and Pattern Recognition, pp 2911–2918

  7. Atreya A, Elkan C (2010) Latent semantic indexing (LSI) fails for TREC collections. SIGKDD Explorations, pp 5–10

  8. Babenko A, Lempitsky VS (2015) Aggregating Deep Convolutional Features for Image Retrieval

  9. Babenko A, Slesarev A, Chigorin A, Lempitsky VS (2014) Neural codes for image retrieval. Computer Vision - ECCV, pp 584–599

  10. Becker CJ, Rigamonti R, Lepetit V, Fua P (2013) Supervised feature learning for curvilinear structure segmentation. Medical Image Computing and Computer-Assisted Intervention - MICCAI, pp 526– 533

  11. Canny J (1986) A computational approach to edge detection. IEEE Transaction on Pattern Analysis Machine Intelligence. pp 679–698

  12. Chatzichristofis SA, Yiannis SB (2008) CEDD color and edge directivity descriptor: A compact descriptor for image indexing and retrieval. Computer Vision Systems, 6th International Conference, pp 312–322

  13. Claveau V, Tavenard R, d’appariement LA (2010) Vectorisation des processus document-requte. Conference en Recherche d’Infomations et Applications. Conference en Recherche d’Infomations et Applications (CORIA), pp 313–324

  14. Delaitre V, Laptev I, Sivic J (2010) Recognizing human actions in still images: a study of bag-of-features and part-based representations. British Machine Vision Conference - BMVC, pp 1–11

  15. Deserno TM, Guld MO, Plodowski B, Spitzer K, Wein BB, Schubert H, Ney H, Seidl T (2008) Extended query refinement for medical image retrieval. Journal Digital Imaging pp 280–289

  16. Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Trevor D (2014) DeCAF A deep convolutional activation feature for generic visual recognition. Proceedings of the 31th International Conference on Machine Learning - ICML, pp 647–655

  17. Douze M, Jegou H, Sandhawalia H, Amsaleg L, Schmid C (2009) Evaluation of GIST descriptors for web-scale image search. Proceedings of the 8th ACM International Conference on Image and Video Retrieval CIVR

  18. Flickner M, Sawhney HS, Ashley J, Huang Q, Dom B, Gorkani M, Hafner J, Lee D, Petkovic D, Steele D, Yanker P (1995) Query by image and video content: The QBIC System. IEEE Computer, pp 23–32

  19. Gong Y, Lazebnik S (2011) Iterative quantization: A procrustean approach to learning binary codes. The 24th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 817–824

  20. Gong Y, Wang L, Guo R, Lazebnik S (2014) Multi-scale orderless pooling of deep convolutional activation features. Computer Vision - ECCV, pp 392–407

  21. Gony J, Cord M, Philipp-Foliguet S, Gosselin Philippe H, Precioso F, Jordan M (2007) RETIN: a smart interactive digital media retrieval system. Proceedings of the 6th ACM International Conference on Image and Video Retrieval, pp 93–96

  22. Hsu W, Rodney Long L, Antani SK (2007) SPIRS: A framework for content-based image retrieval from large biomedical databases. (MEDINFO)- Proceedings of the 12th World Congress on Health (Medical) Informatics - Building Sustainable Health Systems, pp 188–192

  23. Jain AK, Nandakumar K, Ross A (2005) Score norMalization in multimodal biometric systems. Pattern Recognition, pp 2270–2285

  24. Jegou H, Chum O (2012) Negative Evidences and Co-occurences in Image Retrieval: The Benefit of PCA and Whitening. Computer Vision. ECCV 12th European Conference on Computer Vision, pp 774–787

  25. Jegou H, Zisserman A (2014) Triangulation embedding and democratic aggregation for image search. IEEE Conference on Computer Vision and Pattern Recognition - CVPR, pp 3310–3317

  26. Jordan C, Watters CR (2004) Extending the Rocchio relevance feedback algorithm to provide contextual retrieval dvances in Web intelligence. Second International Atlantic Web Intelligence Conference (AWIC), pp 135–144

  27. Kalantidis Y, Mellina C, Osindero S (2016) Cross-dimensional weighting for aggregated deep convolutional features. Computer Vision - ECCV, pp 685–701

  28. Karamti H (2013) Vectorisation du modle d’appariement pour la recherche d’images par le contenu. Conference en Recherche d’Infomations et Applications (CORIA), pp 335–340

  29. Karamti H, Tmar M, Benammar A (2012) A new relevance feedback approach for multimedia retrieval. IKE, pp 129–135

  30. Karamti H, Tmar M, Gargouri F (2014) Content-based image retrieval system using neural network. 11th IEEE/ACS International Conference on Computer Systems and Applications (AICCSA), pp 723–728

  31. Karamti H, Tmar M, Gargouri F (2015) Content-based image retrieval system with relevance feedback. WEBIST 2015 - Proceedings of the 11th International Conference on Web Information Systems and Technologies, pp 287–292

  32. Kasutani E, Yamada A (2001) The MPEG-7 color layout descriptor: a compact image feature description for high-speed image/video segment retrieval. ICIP (1), pp 674–677

  33. Kulkarni S, Srinivasan B, Ramakrishna MV (1999) Vector-space image model (VSIM) for content-based retrieval. 10th International Workshop on Database and Expert Systems Applications, pp 899–903

  34. Lee H, Grosse RB, Ranganath R, Ng AY (2011) Unsupervised learning of hierarchical representations with convolutional deep belief networks Commun. ACM, pp 95–103

  35. Lindeberg T (1996) Edge Detection and Ridge Detection with Automatic Scale Selection. Conference on Computer Vision and Pattern Recognition CVPR, pp 465–470

  36. Liu W, Wang J, Ji R, Jiang Y-G, Chang S-F (2012) Supervised hashing with kernels. Conference on Computer Vision and Pattern Recognition, Providence, RI, pp 2074–2081

    Google Scholar 

  37. Lowe DG (1999) Object recognition from local scale-invariant features. ICCV, pp 1150–1157

  38. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, pp 91–110

  39. Mandal S, Sudarshan VP, Nagaraj Y, Dean-Ben XL, Razansky D (2015) Multiscale edge detection and parametric shape modeling for boundary delineation in optoacoustic images. 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp 707–710

  40. Manning CD, Raghavan P, Schutze H (2008) Introduction to information retrieval. Cambridge University Press

  41. Miao J, Huang JX, Ye Z (2012) Proximity-based rocchio’s model for pseudo relevance. The 35th International ACM-SIGIR conference on research and development in Information Retrieval, pp 535–544

  42. Ming A, Ma H (2007) A blob detector in color images. Proceedings of the 6th ACM International Conference on Image and Video Retrieval, pp 364–370

  43. Ng JY-H, Yang F, Davis LS (2015) Exploiting local features from deep networks for image retrieval. IEEE Conference on Computer Vision and Pattern Recognition Workshops - CVPR, pp 53–61

  44. Ortiz-Jaramillo B, Benitez-Restrepo H, Garcia-Alvarez JC, Castellanos-Dominguez CG (2010) Region of Interest Extraction based on Multiresolution Analysis for Infrared Nondestructive Testing. 10th Quantitative Infrared Thermography Conference QIRT

  45. Park DK, Jeon YS, Won CS (2000) Efficient use of local edge histogram descriptor. Proceedings of the ACM Multimedia 2000 Workshops, pp 51–54

  46. Paulin M, Douze M, Harchaoui Z, Mairal J, Perronnin F, Schmid C (2015) Local convolutional features with unsupervised training for image retrieval. IEEE International Conference on Computer Vision - ICCV, pp 91–99

  47. Philbin J, Chum O, Isard M, Sivic J, Zisserman A (2007) Object retrieval with large vocabularies and fast spatial matching. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR)

  48. Philbin J, Isard M, Sivic J, Zisserman Andrew (2010) Descriptor learning for efficient retrieval. Computer Vision - (ECCV) 2010, 11th European Conference on Computer Vision, pp 677–691

  49. Ramanathan V, Mishra S, Mitra P (2011) Quadtree decomposition based extended vector space model for image retrieval. IEEE Workshop on Applications of Computer Vision WACV, pp 139– 144

  50. Ramos H, Ribeiro AL, Giro PM (1994) A Two-dimensional Vector Model of Ferromagnetic Hysteresis. Journal of Magnetism and Magnetic Materials, pp 574–577

  51. Rosten E, Drummond T (2006) Machine learning for high-speed corner detection. Computer Vision - ECCV, 9th European Conference on Computer Vision, ppp 430–443

  52. Ruthven I, Lalmas M (2003) A survey on the use of relevance feedback for information access systems Knowledge Engineering Review, pp 95–145

  53. Salembier P (2002) Overview of the MPEG-7 Standard and of Future Challenges for Visual Information Analysis. EURASIP Journal of Advanced Signal Proceedings, pp 343–353

  54. Salembier P, Sikora T (2002) Introduction to MPEG-7: Multimedia content description interface. Wiley,

  55. Salton G, Wong A, Yang CS (1975) A Vector Space Model for Automatic Indexing Commun. ACM, pp 613–620

  56. Schettini R, Ciocca G, Gagliardi I (2009) Feature extraction for content-based image retrieval. Encyclopedia of Database Systems, pp 1115–1119

  57. Sheikholeslami G, Chang W, SemQuery AZ (2002) Semantic clustering and querying on heterogeneous features for visual data. IEEE Transactions on Knowledge and Data Engineering, pp 988– 1002

  58. Smeulders AWM, Worring M, Santini S, Gupta A, Jain RC (2000) Content-based image retrieval at the end of the early years. IEEE Transactions Pattern Analysis Machine Intelligence, pp 1349– 1380

  59. Stathopoulos V, Jose JM (2011) Bayesian probabilistic models for image retrieval. Proceedings of the Second Workshop on Applications of Pattern Analysis (WAPA), pp 41–47

  60. Teran L, Mordohai P (2014) 3D interest point detection via discriminative learning. Computer Vision - ECCV - 13th European Conference, pp 159–173

  61. Tolias G (2015) Ronan Sicre and Herve Jegou Particular object retrieval with integral max-pooling of CNN activations

  62. Tsai C-F, Hu Y-H, Chen Z-Y (2015) Factors affecting rocchio-based pseudorelevance feedback in image retrieval. JASIST, pp 40–57

  63. Wang D, Tan X (2014) C-SVDDNet: An Effective Single-Layer Network for Unsupervised Feature Learning

  64. Wang Y, Gong M, Wang T, Cohen-Or D, Zhang H, Chen B (2013) Projective analysis for 3D shape segmentation. ACM Transactions, pp 192–192

  65. Wengert C, Douze M, Jegou H (2011) Bag-of-colors for improved image search. Proceedings of the 19th International Conference on Multimedia, pp 1437–1440

  66. Westerveld T, De Vries AP, Van Ballegooij A, De Jong F, Hiemstra D (2003) A Probabilistic Multimedia Retrieval Model and Its Evaluation. EURASIP Journal of Advanced Signal Proceedings, pp 186– 198

  67. Winder SAJ, Hua G, Brown MA (2009) Picking the best DAISY. IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR, pp 178–185

  68. Yan R, Hauptmann AG, Jin R (2003) Negative pseudo-relevance feedback in content-based video retrieval. Proceedings of the Eleventh ACM International Conference on Multimedia, pp 343– 346

  69. Ye Z, Huang JX (2014) A simple term frequency transformation model for effective pseudo relevance feedback. The 37th International ACM-SIGIR Conference on Research and Development in Information Retrieval, pp 323–332

  70. Yue J, Li Z, Lu L, Fu Z (2011) Content-based image retrieval using color and texture fused features. Mathematical and Computer Modelling, pp 1121–1127

  71. Zhang D, Lu G (2001) Content-Based Shape Retrieval Using Different Shape Descriptors: A Comparative Study. Proceedings of the IEEE International Conference on Multimedia and Expo

Download references

Author information

Authors and Affiliations

  1. National School of Engineers of Sfax, University of Sfax, Sfax, Tunisia

    H. Karamti

  2. Institut Supérieur d’Informatique et de Multimédia de Sfax, Sfax, Tunisia

    M. Tmar & F. Gargouri

  3. Computer Science Lab (L3i), University of La Rochelle, La Rochelle, France

    M. Visani

  4. Vietnam-France, ICT Lab, USTH, Hanoi, France

    M. Visani

  5. University of Poitiers, Poitiers, France

    T. Urruty

Authors
  1. H. Karamti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Tmar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Visani
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. T. Urruty
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. F. Gargouri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to H. Karamti.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karamti, H., Tmar, M., Visani, M. et al. Vector space model adaptation and pseudo relevance feedback for content-based image retrieval. Multimed Tools Appl 77, 5475–5501 (2018). https://doi.org/10.1007/s11042-017-4463-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4463-x

Keywords

Search

Navigation