Skip to main content
Springer Nature Link
Log in

Speed up interactive image retrieval

  • Regular Paper
  • Published:
The VLDB Journal Aims and scope Submit manuscript

Abstract

In multimedia retrieval, a query is typically interactively refined towards the “optimal” answers by exploiting user feedback. However, in existing work, in each iteration, the refined query is re-evaluated. This is not only inefficient but fails to exploit the answers that may be common between iterations. Furthermore, it may also take too many iterations to get the “optimal” answers. In this paper, we introduce a new approach called OptRFS (optimizing relevance feedback search by query prediction) for iterative relevance feedback search. OptRFS aims to take users to view the “optimal” results as fast as possible. It optimizes relevance feedback search by both shortening the searching time during each iteration and reducing the number of iterations. OptRFS predicts the potential candidates for the next iteration and maintains this small set for efficient sequential scan. By doing so, repeated candidate accesses (i.e., random accesses) can be saved, hence reducing the searching time for the next iteration. In addition, efficient scan on the overlap before the next search starts also tightens the search space with smaller pruning radius. As a step forward, OptRFS also predicts the “optimal” query, which corresponds to “optimal” answers, based on the early executed iterations’ queries. By doing so, some intermediate iterations can be saved, hence reducing the total number of iterations. By taking the correlations among the early executed iterations into consideration, OptRFS investigates linear regression, exponential smoothing and linear exponential smoothing to predict the next refined query so as to decide the overlap of candidates between two consecutive iterations. Considering the special features of relevance feedback, OptRFS further introduces adaptive linear exponential smoothing to self-adjust the parameters for more accurate prediction. We implemented OptRFS and our experimental study on real life data sets show that it can reduce the total cost of relevance feedback search significantly. Some interesting features of relevance feedback search are also discovered and discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. http://kdd.ics.uci.edu/databases/corelfeatures

  2. Amer-Yahia, S., Case, P., Roelleke, T., Shanmugasundaram, J., Weikum, G.: Report on the db/ir panel at sigmod 2005. SIGMOD Record. 34(4), 71–74 (2005)

    Article  Google Scholar 

  3. Andoni, A., Indyk, P.: Near-optimal hashing algorithms for near neighbor problem in high dimensions. In: FOCS, pp. 459–468 (2006)

  4. Baeza-Yates, R., Consens, M.: The continued saga of DB-IR integration. In: VLDB Tutotial (2004)

  5. Bang, H., Chen, T.: Feature space warping: an approach to relevance feedback. In: ICIP, pp. I: 968–971 (2002)

  6. Bartolini, I., Ciaccia, P., Waas, F.: Feedbackbypass: a new approach to interactive similarity query processing. In: VLDB, pp. 201–210 (2001)

  7. Benninga, S.: Financial modeling. In: MIT Press (2000)

  8. Böhm, C., Berchtold, S., Keim, D.: Searching in high-dimensional spaces: Index structures for improving the performance of multimedia databases. ACM Comput. Surv. 33(3), 322–373 (2001)

    Article  Google Scholar 

  9. Chaudhuri, S., Ramakrishnan, R., Weikum, G.: Integrating DB and IR technologies: what is the sound of one hand clapping? In: CIDR, pp. 1–12 (2005)

  10. Cheng, J., Wang, K.: Multi-view sampling for relevance feedback in image retrieval. In: ICPR, pp. II: 881–884 (2006)

  11. Croft, W.B., Schek, H.J.: Introduction to the special issue on database and information retrieval integration. VLDB J. 17(1), 1–3 (2008)

    Google Scholar 

  12. El Naqa, I., Yang, Y., Galatsanos, N., Wernick, M.: Relevance feedback based on incremental learning for mammogram retrieval. In: ICIP, pp. I: 729–732 (2003)

  13. Ferecatu, M., Boujemaa, N.: Interactive remote sensing image retrieval using active relevance feedback. IEEE Trans. Geosci. Remote Sensing 45(4), 818–826 (2007)

    Article  Google Scholar 

  14. Franco, A., Lumini, A., Maio, D.: A new approach for relevance feedback through positive and negative samples. In: ICPR, pp. IV: 905–908 (2004)

  15. Hua, K., Yu, N., Liu, D.: A multiple neighborhood approach to relevance feedback in content-based image retrieval. In: ICDE (2006)

  16. Ishikawa, Y., Subramanya, R., Faloutsos, C.: Mindreader: querying databases through multiple examples. In: VLDB, pp. 218–227 (1998)

  17. K. A. Hua, N.Y., Liu, D.Z.: Query decomposition: a multiple neighborhood approach to relevance feedback in content-based image retrieval. In: ICDE, p. 84 (2006)

  18. Kherfi, M., Ziou, D.: Relevance feedback for cbir: a new approach based on probabilistic feature weighting with positive and negative examples. IP 15(4), 1017–1030 (2006)

    Google Scholar 

  19. Kim, D.H., Chung, C.W.: Qcluster: relevance feedback using adaptive clustering for content based image retrieval. In: SIGMOD, pp. 599–610 (2003)

  20. Lejsek, H., Asmundsson, F.H., Jonsson, B.T., Amsaleg, L.: Scalability of local image descriptors: a comparative study. In: ACM Multimedia, pp. 589–598 (2006)

  21. Lu, H., Ooi, B.C., Shen, H.T., Xue, X.: Hierarchical indexing structure for efficient similarity search in video retrieval. IEEE Trans. Knowl. Data Eng. 18(11), 1544–1559 (2006)

    Article  Google Scholar 

  22. Lu, Y., Hu, C., Zhu, X., Zhang, H., Yang, Q.: A unified framework for semantics and feature based relevance feedback in image retrieval systems. In: ACM Multimedia, pp. 31–37 (2000)

  23. Lu, Y., Zhang, H., Liu, W., Hu, C.: Joint semantics and feature based image retrieval using relevance feedback. IEEE Trans. Multimedia 5(3), 339–347 (2003)

    Article  Google Scholar 

  24. Muneesawang, P., Guan, L.: An interactive approach for cbir using a network of radial basis functions. IEEE Trans. Multimedia 6(5), 703–716 (2004)

    Article  Google Scholar 

  25. Oh, S., Chung, M., Sull, S.: Relevance feedback reinforced with semantics accumulation. In: CIVR, pp. 448–454 (2004)

  26. Rao, Y., Mundur, P., Yesha, Y.: Fuzzy svm ensembles for relevance feedback in image retrieval. In: CIVR, pp. 350–359 (2006)

  27. Rui, Y., Huang, T.: Optimizing learning in image retrieval. In: ICCV, pp. 236–243 (2000)

  28. Rui, Y., Huang, T.S., Mehrotra, S.: Content-based image retrieval with relevance feedback in mars. In: ICIP, pp. 815–818 (1997)

  29. Rui, Y., Huang, T.S., Ortega, M., Mehrotra, S.: Relevance feedback: a power tool in interactive content-based image retrieval. IEEE Trans. Circuits Systems Video Technol. 8(5), 644–655 (1998)

    Article  Google Scholar 

  30. Saha, S., Das, A., Chanda, B.: Image retrieval based on indexing and relevance feedback. PRL 28(3), 357–366 (2007)

    Google Scholar 

  31. Shao, J., Huang, Z., Shen, H.T., Zhou, X., Li, Y.: Dynamic batch nearest neighbour search in video retrieval. In: ICDE, pp. 1395–1399 (2007)

  32. Shen, H.T., Ooi, B.C., Zhou, X., Huang, Z.: Towards effective indexing for very large video sequence database. In: SIGMOD, pp. 730–741 (2005)

  33. Tong, S., Chang, E.: Support vector machine active learning for image retrieval. In: ACM Multimedia, pp. 107–118 (2001)

  34. de Ves, E., Domingo, J., Ayala, G., Zuccarello, P.: A novel bayesian framework for relevance feedback in image content-based retrieval systems. PR 39(9), 1622–1632 (2006)

    MATH  Google Scholar 

  35. Wang, L., Gao, Y., Chan, K., Xue, P., Yau, W.: Retrieval with knowledge-driven kernel design: an approach to improving svm-based cbir with relevance feedback. In: ICCV, pp. II: 1355–1362 (2005)

  36. Weber, R., Schek, H., Blott, S.: A quantitative analysis and performance study for similarity search methods in high dimensional spaces. In: VLDB, pp. 194–205 (1998)

  37. Wu, L., Faloutsos, C., Sycara, K., Payne, T.R.: Falcon: feedback adaptive loop for content-based retrieval. In: VLDB, pp. 297–306 (2000)

  38. Wu, P., Manjunath, B.: Adaptive nearest neighbor search for relevance feedback in large image datasets. In: ACM Multimedia, pp. 87–98 (2001)

  39. Yin, P., Bhanu, B., Chang, K., Dong, A.: Integrating relevance feedback techniques for image retrieval using reinforcement learning. PAMI 27(10), 1536–1551 (2005)

    Google Scholar 

  40. Zhang, H., Chen, Z., Li, M., Su, Z.: Relevance feedback and learning in content-based image search. World Wide Web 6(2), 131–155 (2003)

    Article  Google Scholar 

  41. Zhou, X.S., Huang, T.S.: Relevance feedback in image retrieval: a comprehensive review. Multimedia Systems 8(6), 536–544 (2003)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Information Technology and Electrical Engineering, The University of Queensland, Brisbane, Australia

    Heng Tao Shen, Zi Huang & Xiaofang Zhou

  2. Department of Computer Science, Harbin Institute of Technology, Harbin, China

    Shouxu Jiang

  3. Department of Computer Science, National University of Singapore, Singapore, Singapore

    Kian-Lee Tan

Authors
  1. Heng Tao Shen
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Shouxu Jiang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Kian-Lee Tan
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Zi Huang
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Xiaofang Zhou
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Heng Tao Shen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shen, H.T., Jiang, S., Tan, KL. et al. Speed up interactive image retrieval. The VLDB Journal 18, 329–343 (2009). https://doi.org/10.1007/s00778-008-0101-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00778-008-0101-6

Keywords