Skip to main content
SpringerLink
Log in

I-Quest: an intelligent query structuring based on user browsing feedback for semantic retrieval of video data

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In spite of significant improvements in video data retrieval, a system has not yet been developed that can adequately respond to a user’s query. Typically, the user has to refine the query many times and view query results until eventually the expected videos are retrieved from the database. The complexity of video data and questionable query structuring by the user aggravates the retrieval process. Most previous research in this area has focused on retrieval based on low-level features. Managing imprecise queries using semantic (high-level) content is no easier than queries based on low-level features due to the absence of a proper continuous distance function. We provide a method to help users search for clips and videos of interest in video databases. The video clips are classified as interesting and uninteresting based on user browsing. The attribute values of clips are classified by commonality, presence, and frequency within each of the two groups to be used in computing the relevance of each clip to the user’s query. In this paper, we provide an intelligent query structuring system, called I-Quest, to rank clips based on user browsing feedback, where a template generation from the set of interesting and uninteresting sets is impossible or yields poor results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Allen J (1983) Maintaining knowledge about temporal intervals. Commun ACM 26(11):823–843

    Article  Google Scholar 

  2. Babaguchi N, Ohara K, Ogura T (2007) Learning personal preference from viewer’s operations for browsing and its application to baseball video retrieval and summarization. IEEE Trans Multimedia 9(5):1016–1025

    Article  Google Scholar 

  3. Chen Y-S, Shahabi C (2003) Yoda, an adaptive soft classification model: content-based similarity queries and beyond. Multimedia Syst 8:523–535

    Article  Google Scholar 

  4. Cox IJ, Miller ML, Minka TP, Yianilos PN (1998) An optimized interaction strategy for bayesian relevance feedback. In: IEEE conference on computer vision and pattern recognition (CVPR’98), Santa Barbara, 23–25 June 1998

  5. Elmasri R, Navathe SB (2006) Fundamentals of database systems, 5th edn. Addison Wesley, Reading

    Google Scholar 

  6. Fellbaum C (ed) (1998) WordNet: an electronic lexical database (language, speech, and communication). MIT, Cambridge

    Google Scholar 

  7. Hauptmann A, Yan R, Lin W-H, Christel M, Wactlar H (2007) Can high-level concepts fill the semantic gap in video retrieval? A case study with broadcast news. IEEE Trans Multimedia 9(5):958–966

    Article  Google Scholar 

  8. He X, Ma W-Y, Li M, Zhang H-J (2003) Learning a semantic space from user’s relevance feedback for image retrieval. IEEE Trans Circuits Syst Video Technol 13(1):1032–1046

    Google Scholar 

  9. Heesch D, Howarth P, Magalhaes J, May A, Pickering M, Yavlinsky A, Ruger S (2004) Video retrieval using search and browsing. In: Proceedings of TRECVID2004

  10. Hoi S, Lyu M (2008) A multimodal and multilevel ranking scheme for large-scale video retrieval. IEEE Trans Multimedia 10(4):607–619

    Article  Google Scholar 

  11. Kelly D, Belkin NJ (2004) Display time as implicit feedback: understanding task effects. In: SIGIR ’04: proceedings of the 27th annual international ACM SIGIR conference on research and development in information retrieval. ACM, New York, pp 377–384

    Google Scholar 

  12. Lew MS, Sebe N, Djeraba C, Jain R (2006) Content-based multimedia information retrieval: state of the art and challenges. ACM Trans Multimedia Comput Commun Appl 2(1):1–19

    Article  Google Scholar 

  13. Martinez JM (2002) Mpeg-7: overview of mpeg-7 descrition tools, part 2. IEEE Multimed 9(3):83–93

    Article  Google Scholar 

  14. Martinez JM, Koenen R, Pereira F (2002) Mpeg-7: the generic multimedia content description standard, part 1. IEEE Multimed 9(2):78–87

    Article  Google Scholar 

  15. Mezaris Y, Doulaverakis H, Herrmann S, O’Connor BLN, Kompatsiaris I, Strintzis GM (2004) Combining textual and visual information processing for interactive video retrieval: schema’s participation to trecvid2004. In: Proceedings of TRECVID2004

  16. MPEG7:XM (2006) http://www.lis.ei.tum.de/research/bv/topics/mmdb/e_mpeg7.html

  17. Mu X (2006) Content-based video retrieval: does video’s semantic visual feature matter? In: SIGIR ’06: proceedings of the 29th annual international ACM SIGIR conference on research and development in information retrieval. ACM, New York, pp 679–680

  18. Munesawang P, Guan L (2005) Adaptive video indexing and automatic/semi-automatic relevance feedback. IEEE Trans Circuits Syst Video Technol 15(8):1032–1046

    Article  Google Scholar 

  19. Ortega M, Rui Y, Chakrabarti K, Porkaew K, Mehrotra S, Huang TS (1998) Supporting ranked Boolean similarity queries in MARS. IEEE Trans Knowl Data Eng 10(6):905–925

    Article  Google Scholar 

  20. Ortega-Binderberger M, Mehrotra S (2003) Handbook of video databases: design and applications. Ch. relevance feedback in multimedia databases. CRC, Boca Raton, pp 511–535

    Google Scholar 

  21. Rui Y, Huang T, Mehrotra S (1997) Content-based image retrieval with relevance feedback in mars. http://citeseer.nj.nec.com/rui97contentbased.html

  22. Rui Y, Huang TS, Mehrotra S, Ortega M (1997) Automatic matching tool selection via relevance feedback in mars. In: Proceedings of the second international conference on visual information systems, San Diego, 15–17 December 1997, pp 109–116

  23. Rui Y, Huang TS, Mehrotra S, Ortega M (1997) A relevance feedback architecture in content-based multimedia information retrieval systems. In: Proceedings of IEEE workshop on content-based access of image and video libraries, in conjunction with CVPR’97, Puerto Rico, June 1997, pp 82–89

  24. Rui Y, Huang TS, Ortega M, Mehrotra S (1998) Relevance feedback: a power tool in interactive content-based image retrieval, special issue on segmentation, description, and retrieval of video content. IEEE Trans Circuits Syst Video Technol 8(5):644–655

    Article  Google Scholar 

  25. Salton G, Buckley C (1990) Improving retrieval performance by relevance feedback. J Am Soc Inf Sci 41(4):288–297

    Article  Google Scholar 

  26. Salton G, Fox E, Wu H (1983) Extended Boolean information retrieval. Commun ACM 26(12):1022–1036

    Article  MATH  MathSciNet  Google Scholar 

  27. Shao J, Huang Z, Shen HT, Zhou X, Lim E-P, Li Y (2008) Batch nearest neighbor search for video retrieval. IEEE Trans Multimedia 10(3):409–420

    Article  Google Scholar 

  28. Shen X, Tan B, Zhai C (2005) Context-sensitive information retrieval using implicit feedback. In: SIGIR ’05: proceedings of the 28th annual international ACM SIGIR conference on research and development in information retrieval. ACM, New York, pp 43–50

    Chapter  Google Scholar 

  29. Snoek CGM, Worring M, Koelma DC, Smeulders AWM (2007) A learned lexicon-driven paradigm for interactive video retrieval. IEEE Trans Multimedia 9(2):280–292

    Article  Google Scholar 

  30. Syeda-Mahmood T, Ponceleon D (2001) Learning video browsing behavior and its application in the generation of video previews. In: Proceedings of the 9th ACM international conference on multimedia, Ottawa, 30 September–5 October 2001, pp 119–128

  31. Taycher L, La Cascia M, Sclaroff S (1997) Image digestion and relevance feedback in the imagerover www search engine. In: Proceedings of international conference on visual information, San Diego, 15–17 December 1997

  32. Wang R, Naphade M, Huang T (2001) Video retrieval and relevance feedback in the context of a post-integration model. In: 2001 IEEE fourth int. workshop on multimedia signal processing. IEEE, Piscataway, pp 33–38

    Chapter  Google Scholar 

  33. Wu L, Faloutos C, Sycara K, Payne T, McCarthy DR (2000) FALCON: feedback adaptive loop for content-based retrieval. In: Proceedings of the 26th international conference on very large data bases, Cairo, 10–14 September 2000, pp 297–306

  34. Xu C, Wang J, Lu H, Zhang Y (2008) A novel framework for semantic annotation and personalized retrieval of sports video. IEEE Trans Multimedia 10(3):421–436

    Article  Google Scholar 

  35. Yin P-Y, Chang K-C, Dong A (2005) Integrating relevance feedback techniques for image retrieval using reinforcement learning. IEEE Trans Pattern Anal Mach Intell 27(10):1536–1551 (fellow-Bir Bhanu)

    Article  Google Scholar 

  36. Zadeh L (1971) Similarity relations and fuzzy orderings. Inf Sci 3:177–200

    Article  MATH  MathSciNet  Google Scholar 

  37. Zhang Y, Zhang X, Xu C, Lu H (2007) Personalized retrieval of sports video. In: MIR ’07: proceedings of the international workshop on workshop on multimedia information retrieval. ACM, New York, pp 313–322

    Chapter  Google Scholar 

Download references

Acknowledgements

We would like to thank Parthan Ulagarakshagan for implementing the baseball video database application.

Author information

Authors and Affiliations

  1. Broadband Enterprises, New York, NY, USA

    Tarun Yadav

  2. Computer Science Department, Technology Hall N360, University of Alabama in Huntsville, Huntsville, AL, 35899, USA

    Tarun Yadav & Ramazan Savaş Aygün

Authors
  1. Tarun Yadav
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ramazan Savaş Aygün
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ramazan Savaş Aygün.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yadav, T., Aygün, R.S. I-Quest: an intelligent query structuring based on user browsing feedback for semantic retrieval of video data. Multimed Tools Appl 43, 145–178 (2009). https://doi.org/10.1007/s11042-009-0262-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-009-0262-3

Keywords

Search

Navigation