Skip to main content
SpringerLink
Log in

Ten Years of Relevance Score for Content Based Image Retrieval

  • Conference paper
  • First Online:
Machine Learning and Data Mining in Pattern Recognition (MLDM 2018)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10935))

Abstract

After more than 20 years of research on Content-Based Image Retrieval (CBIR), the community is still facing many challenges to improve the retrieval results by filling the semantic gap between the user needs and the automatic image description provided by different image representations. Including the human in the loop through Relevance Feedback (RF) mechanisms turned out to help improving the retrieval results in CBIR. In this paper, we claim that Nearest Neighbour approaches still provide an effective method to assign a Relevance Score to images, after the user labels a small set of images as being relevant or not to a given query. Although many other approaches to relevance feedback have been proposed in the past ten years, we show that the Relevance Score, while simple in its implementation, allows attaining superior results with respect to more complex approaches, can be easily adopted with any feature representations. Reported results on different real-world datasets with a large number of classes, characterised by different degrees of semantic and visual intra- e inter-class variability, clearly show the current challenges faced by CBIR system in reaching acceptable retrieval performances, and the effectiveness of Nearest neighbour approaches to exploit Relevance Feedback.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www.vision.caltech.edu/Image_Datasets/.

  2. 2.

    http://www.robots.ox.ac.uk/~vgg/data/flowers/.

References

  1. Arevalillo-Herráez, M., Domingo, J., Ferri, F.J.: Combining similarity measures in content-based image retrieval. Pattern Recogn. Lett. 29(16), 2174–2181 (2008)

    Article  Google Scholar 

  2. Arevalillo-Herráez, M., Ferri, F.J., Domingo, J.: A naive relevance feedback model for content-based image retrieval using multiple similarity measures. Pattern Recogn. 43(3), 619–629 (2010)

    Article  Google Scholar 

  3. Babenko, A., Slesarev, A., Chigorin, A., Lempitsky, V.: Neural codes for image retrieval. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 584–599. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_38

    Chapter  Google Scholar 

  4. Rosdi, B.A., Shing, C.W., Suandi, S.A.: Finger vein recognition using local line binary pattern. Sensors 11, 11357–11371 (2011)

    Article  Google Scholar 

  5. Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)

    Article  Google Scholar 

  6. Chatzichristofis, S.A., Boutalis, Y.S.: CEDD: color and edge directivity descriptor: a compact descriptor for image indexing and retrieval. In: Gasteratos, A., Vincze, M., Tsotsos, J.K. (eds.) ICVS 2008. LNCS, vol. 5008, pp. 312–322. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-79547-6_30

    Chapter  Google Scholar 

  7. Chen, H.: A socio-technical perspective of museum practitioners’ image-using behaviors. The Electron. Libr. 25(1), 18–35 (2007)

    Article  Google Scholar 

  8. Chen, Y., Zhou, X.S., Huang, T.: One-class SVM for learning in image retrieval. ICIP 1, 34–37 (2001)

    Google Scholar 

  9. Cohn, D.A., Atlas, L.E., Ladner, R.E.: Improving generalization with active learning. Mach. Learn. 15(2), 201–221 (1994)

    Google Scholar 

  10. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of CVPR, pp. 886–893 (2005)

    Google Scholar 

  11. Dang-Nguyen, D.T., Piras, L., Giacinto, G., Boato, G., De Natale, F.G.B.: Multimodal retrieval with diversification and relevance feedback for tourist attraction images. ACM Trans. Multimedia Comput. Commun. Appl. 13(4), 49:1–49:24 (2017)

    Article  Google Scholar 

  12. Datta, R., Joshi, D., Li, J., Wang, J.Z.: Image retrieval: ideas, influences, and trends of the new age. ACM Comput. Surv. 40(2), 1–60 (2008)

    Article  Google Scholar 

  13. Escalante, H.J., Hérnadez, C.A., Sucar, L.E., Montes, M.: Late fusion of heterogeneous methods for multimedia image retrieval. In: Proceedings of the 1st ACM International Conference on Multimedia Information Retrieval, pp. 172–179 (2008)

    Google Scholar 

  14. Giacinto, G.: A nearest-neighbor approach to relevance feedback in content based image retrieval. In: Proceedings of the 6th ACM International Conference on Image and Video Retrieval, CIVR 2007, pp. 456–463. ACM, New York (2007)

    Google Scholar 

  15. Giacinto, G., Roli, F.: Bayesian relevance feedback for content-based image retrieval. Pattern Recogn. 37(7), 1499–1508 (2004)

    Article  Google Scholar 

  16. Giacinto, G., Roli, F.: Nearest-prototype relevance feedback for content based image retrieval. In: ICPR, vol. 2, pp. 989–992 (2004)

    Google Scholar 

  17. Graps, A.: An introduction to wavelets. IEEE Comput. Sci. Eng. 2(2), 50–61 (1995)

    Article  Google Scholar 

  18. Hoi, S.C.H., Jin, R., Zhu, J., Lyu, M.R.: Semisupervised SVM batch mode active learning with applications to image retrieval. ACM Trans. Inf. Syst. 27(3), 16:1–16:29 (2009)

    Article  Google Scholar 

  19. Jégou, H., Zisserman, A.: Triangulation embedding and democratic aggregation for image search. In: Proceedings of CVPR, pp. 3310–3317. IEEE Computer Society (2014)

    Google Scholar 

  20. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: 26th Annual Conference on Advances in Neural Information Processing Systems, pp. 1106–1114 (2012)

    Google Scholar 

  21. Laaksonen, J., Koskela, M., Oja, E.: PicSOM-self-organizing image retrieval with MPEG-7 content descriptors. IEEE Trans. Neural Netw. 13(4), 841–853 (2002)

    Article  Google Scholar 

  22. Lee, T.S.: Image representation using 2D Gabor wavelets. IEEE Trans. Pattern Anal. Mach. Intell. 18(10), 959–971 (1996)

    Article  Google Scholar 

  23. van Leuken, R.H., Garcia, L., Olivares, X., van Zwol, R.: Visual diversification of image search results. In: ACM International Conference on World Wide Web, pp. 341–350 (2009)

    Google Scholar 

  24. Liang, S., Sun, Z.: Sketch retrieval and relevance feedback with biased SVM classification. Pattern Recogn. Lett. 29(12), 1733–1741 (2008)

    Article  Google Scholar 

  25. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  26. Mitro, J.: Content-based image retrieval tutorial. ArXiv e-prints (2016)

    Google Scholar 

  27. Müller, H., Clough, P.D., Deselaers, T., Caputo, B. (eds.): ImageCLEF: Experimental Evaluation in Visual Information Retrieval. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15181-1

    Book  MATH  Google Scholar 

  28. Ojala, T., Pietikäinen, M., Mäenpää, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)

    Article  Google Scholar 

  29. Passerini, A., Pontil, M., Frasconi, P.: New results on error correcting output codes of kernel machines. IEEE Trans. Neural Netw. 15(1), 45–54 (2004)

    Article  Google Scholar 

  30. Pavlidis, T.: Limitations of content-based image retrieval. Technical report, Stony Brook University (2008)

    Google Scholar 

  31. Piras, L., Giacinto, G.: Neighborhood-based feature weighting for relevance feedback in content-based retrieval. In: WIAMIS, pp. 238–241. IEEE Computer Society (2009)

    Google Scholar 

  32. Piras, L., Giacinto, G.: Information fusion in content based image retrieval: a comprehensive overview. Inf. Fusion 37, 50–60 (2017)

    Article  Google Scholar 

  33. Piras, L., Giacinto, G., Paredes, R.: Enhancing image retrieval by an exploration-exploitation approach. In: Perner, P. (ed.) MLDM 2012. LNCS (LNAI), vol. 7376, pp. 355–365. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31537-4_28

    Chapter  Google Scholar 

  34. Piras, L., Giacinto, G., Paredes, R.: Passive-aggressive online learning for relevance feedback in content based image retrieval. In: Proceedings of the 2nd International Conference on Pattern Recognition Applications and Methods, pp. 182–187 (2013)

    Google Scholar 

  35. Piras, L., Tronci, R., Giacinto, G.: Diversity in ensembles of codebooks for visual concept detection. In: Petrosino, A. (ed.) ICIAP 2013. LNCS, vol. 8157, pp. 399–408. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-41184-7_41

    Chapter  Google Scholar 

  36. Rocchio, J.J.: Relevance feedback in information retrieval, pp. 313–323. Prentice Hall, Englewood Cliffs (1971)

    Google Scholar 

  37. Rui, Y., Huang, T.S., Mehrotra, S.: Content-based image retrieval with relevance feedback in MARS. In: International Conference on Image Processing Proceedings, pp. 815–818, October 1997

    Google Scholar 

  38. Rui, Y., Huang, T.S., Mehrotra, S.: Relevance feedback: a power tool in interactive content-based image retrieval. IEEE Trans. Circuits Syst. Video Technol. 8(5), 644–655 (1998)

    Article  Google Scholar 

  39. da Torres, R.S., Falcão, A.X., Gonçalves, M.A., Papa, J.P., Zhang, B., Fan, W., Fox, E.A.: A genetic programming framework for content-based image retrieval. Pattern Recogn. 42(2), 283–292 (2009)

    Article  Google Scholar 

  40. Sivic, J., Zisserman, A.: Efficient visual search for objects in videos. Proc. IEEE 96(4), 548–566 (2008)

    Article  Google Scholar 

  41. Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1349–1380 (2000)

    Article  Google Scholar 

  42. Tong, S., Chang, E.Y.: Support vector machine active learning for image retrieval. In: ACM Multimedia, pp. 107–118 (2001)

    Google Scholar 

  43. Tronci, R., Murgia, G., Pili, M., Piras, L., Giacinto, G.: ImageHunter: a novel tool for relevance feedback in content based image retrieval. In: Lai, C., Semeraro, G., Vargiu, E. (eds.) New Challenges in Distributed Information Filtering and Retrieval. SCI, vol. 439, pp. 53–70. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-31546-6_4

    Chapter  Google Scholar 

  44. Tsai, C.M., Qamra, A., Chang, E., Wang, Y.F.: Extent: interring image metadata from context and content. In: IEEE International Conference on Multimedia and Expo, pp. 1270–1273 (2006)

    Google Scholar 

  45. Yu, J., Qin, Z., Wan, T., Zhang, X.: Feature integration analysis of bag-of-features model for image retrieval. Neurocomputing 120, 355–364 (2013)

    Article  Google Scholar 

Download references

Acknowledgements

This work has been supported by the Regional Administration of Sardinia (RAS), Italy, within the project BS2R - Beyond Social Semantic Recommendation (POR FESR 2007/2013 - PIA 2013).

Author information

Authors and Affiliations

  1. Pluribus One, Cagliari, Italy

    Luca Piras & Giorgio Giacinto

  2. Department of Electrical and Electronic Engineering, University of Cagliari, Piazza d’Armi, 09123, Cagliari, Italy

    Lorenzo Putzu, Luca Piras & Giorgio Giacinto

Authors
  1. Lorenzo Putzu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luca Piras
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Giorgio Giacinto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lorenzo Putzu .

Editor information

Editors and Affiliations

  1. Institute of Computer Vision and Applied Computer Sciences, Leipzig, Germany

    Petra Perner

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Putzu, L., Piras, L., Giacinto, G. (2018). Ten Years of Relevance Score for Content Based Image Retrieval. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2018. Lecture Notes in Computer Science(), vol 10935. Springer, Cham. https://doi.org/10.1007/978-3-319-96133-0_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-96133-0_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-96132-3

  • Online ISBN: 978-3-319-96133-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation