Skip to main content
SpringerLink
Log in

A novel relevance feedback method for CBIR

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

In this paper, we address the challenge about insufficiency of training set and limited feedback information in each relevance feedback (RF) round during the process of content based image retrieval (CBIR). We propose a novel active learning scheme to utilize the labeled and unlabeled images to build the initial Support Vector Machine (SVM) classifier for image retrieving. In our framework, two main components, a pseudo-label strategy and an improved active learning selection method, are included. Moreover, a feature subspace partition algorithm is proposed to model the retrieval target from users by the analysis from relevance labeled images. Experimental results demonstrate the superiority of the proposed method on a range of databases with respect to the retrieval accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8

Similar content being viewed by others

References

  1. Bezdek, J.C.: Convergence theorem for the fuzzy ISODATA clustering algorithms. IEEE Trans. Pattern Anal. Mach. Intell. PAMI-2(1), 1–8 (2009)

    Article  Google Scholar 

  2. Cao, X., Zhang, C., Fu, H., Guo, X.J.: Saliency-aware nonparametric foreground annotation based on weakly labeled data. IEEE Trans. Neural Netw. Learn. Syst. 27(6), 1253–1265 (2016)

    Article  MathSciNet  Google Scholar 

  3. Demir, B., Bruzzone, L.: A novel active learning method in relevance feedback for content-based remote sensing image retrieval. IEEE Trans. Geosci. Remote Sens. 53(5), 2323–2334 (2015)

    Article  Google Scholar 

  4. Elhamifar, E., Sapiro, G., Yang, A., Satry, S.S.: A convex optimization framework for active learning. In: IEEE International Conference on Computer Vision (ICCV). IEEE Computer Society, pp. 209– 216 (2013)

  5. Feng, J., Wei, Y., Tao, L., Zhang, C., Sun, J.: Salient object detection by composition. In: Proc IEEE Int. Conf. Comput. Vis., pp. 1028–1035 (2011)

  6. Gao, X., Xiao, B., Tao, D., Li, X.: Image categorization: Graph edit distance+edge direction histogram. Pattern Recogn. 41(10), 3179–3191 (2008)

    Article  Google Scholar 

  7. Hoi, S. C.H., Jin, R., Zhu, J., Lyu, M. R.: Semi-supervised SVM Batch mode active learning with applications to image retrieval. Acm Trans. Inf. Syst. 27(3), 16–32 (2009)

    Article  Google Scholar 

  8. Hu, R.Y., Zhu, X.F., Cheng, D.B., He, W., Yan, Y., Song, J.K., Zhang, S.C.: Graph self-representation method for unsupervised feature selection. Neurocomputing 220, 130–137 (2017)

    Article  Google Scholar 

  9. Hu, M.Q., Yang, Y., Shen, F.M., Zhang, L.M., Shen, H.T., Li X.L.: Robust Web Image Annotation via Exploring Multi-facet and Structural Knowledge. IEEE Transactions on Image Processing 26(10), 4871–4884 (2017)

    Article  MathSciNet  Google Scholar 

  10. Hu, M.Q., Yang, Y., Shen, F., Xie, N., Shen, H.T.: Hashing with Angular Reconstructive Embeddings. IEEE Transactions on Image Processing 27(2), 545–555 (2018)

    Article  MathSciNet  Google Scholar 

  11. Jegou, H., Douze, M., Schmid, C.: Hamming embedding and weak geometric consistency for large scale image search. Euro. Conf. Comput. Vis. 5302, 304–317 (2008)

    Google Scholar 

  12. Jones, S., Shao, L., Zhang, J., Liu, Y.: Relevance feedback for real-world human action retrieval. Pattern Recogn. Lett. 33(4), 446–452 (2012)

    Article  Google Scholar 

  13. Kim, D.W., Lee, K.Y., Lee, D.: kernel-based subtractive clustering method. A Pattern. Recogn. Lett. 26(7), 879–891 (2005)

    Article  Google Scholar 

  14. Krizhevsky, A.: Learning multiple layers of features from tiny images (2009)

  15. Li, J., Wang, J.Z.: Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Trans. Pattern Anal. Mach. Intell. 25(9), 1075–1088 (2003)

    Article  Google Scholar 

  16. Li, J., Allinson, N.M.: Relevance feedback in content-based image retrieval: a survey. In: Handbook on Neural Information Processing, Springer, Berlin, Heidelberg, pp. 433–469 (2013)

    Google Scholar 

  17. Lin, K., Yang, H.F., Hsiao, J.H., Chen, C.S.: Deep learning of binary hash codes for fast image retrieval, Computer Vision and Pattern Recognition Workshops (2015)

  18. Liu, R.J., Wang, Y.H., Baba, T., Masumoto, D., Nagata, S.: SVM-Based active feedback in image retrieval using clustering and unlabeled Data. Pattern Recogn. 41(8), 2645–2655 (2008)

    Article  Google Scholar 

  19. Liu, H., Wang, R., Shan, S., Chen, X.: Deep supervised hashing for fast image retrieval. In: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE, pp. 2064–2072 (2016)

  20. Luo, Y.D., Yang, Y., Shen, F.M., Huang, Z., Zhou, P., Shen, H.T.: Robust discrete code modeling for supervised hashing. Pattern Recognition 75, 128–135 (2018)

    Article  Google Scholar 

  21. Manjunath, B.S., Ma, W.Y.: Texture features for browsing and retrieval of image data. IEEE Trans. Pattern Anal. Mach. Intell. 18, 837–842 (1996)

    Article  Google Scholar 

  22. Markus, S., Markus, O.: Similarity of color images, SPIE Storage and Retrieval for Image and Video Databases (1995)

  23. Min, R., Cheng, H.D.: Effective image retrieval using dominant color descriptor and fuzzy support machine. Pattern Recogn. 4(12), 147–157 (2009)

    Article  Google Scholar 

  24. Niblack, W., Barber, R., Equitz, W., Fickner, M., Glasman, E., Petkovic, D., Yanker, P.: The QBIC project: Querying images by content using color, texture and shape. In: Proceedings of the SPIE Conference on GeometricMethods in Computer Vision II, San Diego, California, USA (1993)

  25. Niu, B., Cheng, J., Bai, X., Lu, H.Q.: Asymmetric propagation based batch mode active learning for image retrieval. Signal Process. 93(6), 1639–1650 (2013)

    Article  Google Scholar 

  26. Qin, T., Zhang, X.D., Liu, T.Y., Wang, D.S., Ma, W.Y., Zhang, H. J.: An active feedback framework for image retrieval. Pattern Recogn. Lett. 29(5), 637–646 (2008)

    Article  Google Scholar 

  27. Rahman, M.M., Antani, S.K., Thoma, G.R.: A learning-based similarity fusion and filtering approach for biomedical image retrieval Using SVM classication and relevance feedback. IEEE Trans. Inf. Technol. Biomed. 15(4), 640–646 (2011)

    Article  Google Scholar 

  28. Rocchio, J.: Relevance feedback in information retrieval, Computer Science (2000)

  29. Russell, B., Torralba, A., Liu, C., Fergus, R., Freeman, W.T.: Object recognition by scene alignment. In: Proc. Adv. Neural Inf. Process. Syst. (NIPS), pp. 1241–1248 (2007)

  30. Samadi, A., Lillicrap, T.P., Tweed, D.B.: Deep learning with dynamic spiking neurons and fixed feedback weights. Neural Comput. 29(3), 578 (2017)

    Article  Google Scholar 

  31. Schmid, C.: Weakly supervised learning of visual models and its application to content-based retrieval. Int. J. Comput. Vis. 56(1), 7–16 (2004)

    Article  Google Scholar 

  32. Schohn, G., Cohn, D.: Less is more: active learning with Support Vector Machines. In: Proc 17th International Conf. Mach. Learn., Stanford, CA, USA, pp. 839–846 (2000)

  33. Scott, G., Klaric, M., Davis, C., Shyu, C.R.: Entropy-balanced bitmap tree for shape-based object retrieval from large-scale satellite imagery databases. IEEE Trans. Geosci. Remote Sens. 49(5), 1603–1616 (2011)

    Article  Google Scholar 

  34. Shi, B., Bai, X., Yao, C.: An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39(11), 2298–2304 (2017)

    Article  Google Scholar 

  35. Tao, D.C., Tang, X.O., Li, X.L., Wu, X.D.: Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 28(7), 1088–1099 (2006)

    Article  Google Scholar 

  36. Tong, S., Chang, E.: Support vector machine active learning for image retrieval. In: Proceedings of the ACM Multimedia, pp. 107–118 (2001)

  37. Wan, J., Wang, D., Hoi, S.C.H., Wu, P., Zhu, J., Zhang, Y.D., Li, J.T.: Deep learning for content-based image retrieval: a comprehensive study. In: ACM International Conference on Multimedia. ACM, pp. 157–166 (2014)

  38. Wang, X.Y., Chen, J.W., Yang, H.Y.: A new integrated SVM classers for relevance feedback content-based image retrieval using EM parameter estimation. Appl. Soft Comput. 4(11), 2787–2804 (2011)

    Article  Google Scholar 

  39. Wang, X., Zhang, B., Yang, H.: Active SVM-based relevance feedback using multiple classifiers ensemble and features re-weighting. Eng. Appl. Artif. Intell. 26, 368–381 (2013)

    Article  Google Scholar 

  40. Wang, X.Y., Li, Y.W., Yang, H.Y., Chen, J.W.: An image retrieval scheme with relevance feedback using feature reconstruction and SVM reclassication. Neurocomputing 127, 214–230 (2014)

    Article  Google Scholar 

  41. Wang, X.Y., Liang, L.L., Li, W.Y., Li, D.M., Yang, H.Y.: A new SVM-based relevance feedback image retrieval using probabilistic feature and weighted kernel function. J. Vis. Commun. Image Represent. 38, 256–275 (2016)

    Article  Google Scholar 

  42. Weir, J.P.: Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. J. Strength Cond. Res. 19(1), 231 (2005)

    Google Scholar 

  43. Wu, K., Yap, K.H.: Fuzzy SVM for content-based image retrieval. IEEE Comput. Intell. Mag. 1(2), 10–16 (2005)

    Google Scholar 

  44. Xia, R., Pan, Y., Lai, H., Liu, C., Yan, S.: Supervised hashing for image retrieval via image representation learning. In: Proceedings of the Twenty-Eighth AAAI Conf. on Artificial Intelligence, vol. 3, pp. 2156–2162 (2014)

  45. Yang, Y., Shen, F.M., Shen, H.T, Li, H.X, Li, X.L: Robust discrete spectral hashing for large-scale image semantic indexing. IEEE Transactions on Big Data 1(4), 162–171 (2015)

    Article  Google Scholar 

  46. Yang, Y., Ma, Z.G., Yang, Y., Nie, F.P., Shen, H.T.: Multitask spectral clustering by exploring intertask correlation. 45(5), 1083–1094 (2015)

  47. Yang, Y., Shen, F.M., Huang, Z., Shen, H.T., Li, X.L.: Discrete nonnegative spectral clustering. IEEE Transactions on Knowledge and Data Engineering 29(9), 1834–1845 (2017)

    Article  Google Scholar 

  48. Zhang, Z., Ji, R.R., Yao, H.X., Xu, P.F., Wang, J.C.: Random sampling SVM based soft query expansion for image retrieval. In: IEEE International Conference on Image and Graphics, Chengdu, Sichuan, China, pp. 805–809 (2007)

  49. Zhu, X.F., Zhang, L., Huang, Z.: A sparse embedding and least variance encoding approach to hashing. IEEE transactions on image processing 23(9), 3737–3750 (2014)

    Article  MathSciNet  Google Scholar 

  50. Zhu, X.F., Li, X.L., Zhang, S.C.: Block-row sparse multiview multilabel learning for image classification. IEEE transactions on cybernetics 46(2), 450–461 (2016)

    Article  Google Scholar 

  51. Zhu, X.F., Li, X.L., Zhang, S.C., Ju, C.H., Wu, X.D.: Robust joint graph sparse coding for unsupervised spectral feature selection. 27(2), 545–555 (2018)

Download references

Acknowledgments

This work was supported by the Fundamental Research Funds for the Central Universities of China(No.A03013023001050,No.ZYGX2016J095).the National Natural Science Foundation of Sichuan(No.2017JY0229).

Author information

Authors and Affiliations

  1. School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, 610054, People’s Republic of China

    Yunbo Rao, Wei Liu, Bojiang Fan & Jiali Song

  2. School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, 611731, People’s Republic of China

    Yang Yang

Authors
  1. Yunbo Rao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bojiang Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiali Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Liu.

Additional information

This article belongs to the Topical Collection: Special Issue on Deep Mining Big Social Data

Guest Editors: Xiaofeng Zhu, Gerard Sanroma, Jilian Zhang, and Brent C. Munsell

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rao, Y., Liu, W., Fan, B. et al. A novel relevance feedback method for CBIR. World Wide Web 21, 1505–1522 (2018). https://doi.org/10.1007/s11280-017-0523-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-017-0523-4

Keywords

Search

Navigation