Skip to main content
SpringerLink
Log in

How to search in MPEG-7 based semantic descriptions: an evaluation of metrics

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

MPEG-7 is an extensive multimedia metadata standard covering an impressive range of aspects of metadata. However, as with most metadata standards details of usage and application of the standards are—at least partially—by design open to interpretation. In case of MPEG-7, storage and transmission of high level metadata on concept level are well defined but retrieval methods are not proposed by the standard. So if for instance a user annotates photos using the MPEG-7 semantic description scheme, there are no standardized ways to index and retrieve the photos based on the annotation. In this article we revisit metrics for relevance assessment based on the MPEG-7 Semantic Description Scheme in the context of information retrieval. We evaluate them in a digital photo retrieval scenario and investigate correlation of similarity and distance metrics to user perception in a user study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Allan J, Aslam J, Belkin N, Buckley C, Callan J, Croft B, Dumais S, Fuhr N, Harman D, Harper DJ, Hiemstra D, Hofmann T, Hovy E, Kraaij W, Lafferty J, Lavrenko V, Lewis D, Liddy L, Manmatha R, McCallum A, Ponte J, Prager J, Radev D, Resnik P, Robertson S, Rosenfeld R, Roukos S, Sanderson M, Schwartz R, Singhal A, Smeaton A, Turtle H, Voorhees E, Weischedel R, Xu J, Zhai C (2003) Challenges in information retrieval and language modeling: report of a workshop held at the center for intelligent information retrieval. University of Massachusetts Amherst, September 2002. SIGIR Forum 37(1):31–47

    Article  Google Scholar 

  2. Athanasiadis T, Avrithis Y (2004) Adding semantics to audiovisual content: the faethon project. In: Image and video retrieval: third international conference, CIVR 2004. LNCS, vol 3115. Springer, Dublin, Ireland, pp 665–673

    Google Scholar 

  3. Baeza-Yates RA, Ribeiro-Neto B (1999) Modern information retrieval. Addison-Wesley Longman Publishing Co., Inc

  4. Berners-Lee T (2001) Conceptual graphs and the semantic web, W3C design issues. 10.1145/1631272.1631456, URL:http://www.w3.org/DesignIssues/CG.html. Accessed 4 Feb 2011

  5. Berretti S, del Bimbo A, Vicario E (2001) Efficient matching and indexing of graph models in content-based retrieval. IEEE Trans Pattern Anal Mach Intell 23(10):1089–1105

    Article  Google Scholar 

  6. Boll S (2007) Multitube-where multimedia and web 2.0 could meet. IEEE Multimed 14(01):9–13

    Article  Google Scholar 

  7. Bunke H, Shearer K (1998) A graph distance metric based on the maximal common subgraph. Pattern Recogn Lett 19(3–4):255–259

    Article  MATH  Google Scholar 

  8. Chadwick BA, Bahr HM, Albrecht SL (1984) Social science research methods. Prentice Hall

  9. Chang SF, Sikora T, Puri A (2001) Overview of the mpeg-7 standard. IEEE Trans Circuits Syst Video Technol 11(6):688–695

    Article  Google Scholar 

  10. Corby O, Dieng R, Hebert C (2000) A conceptual graph model for w3c resource description framework. In: ICCS, Springer, Darmstadt, Germany, pp 468–482

  11. Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: Ideas, influences, and trends of the new age. ACM Comput Surv 40(2):1–60

    Article  Google Scholar 

  12. Dickinson PJ, Bunke H, Dadej A, Kraetzl M (2003) On graphs with unique node labels, vol 2726

  13. Doeller M, Kosch H, Doerflinger B, Bachlechner A, Blaschke G (2002) Demonstration of an mpeg-7 multimedia data cartridge. In: MULTIMEDIA ’02: proceedings of the tenth ACM international conference on Multimedia, ACM Press, New York, NY, USA, pp 85–86

    Chapter  Google Scholar 

  14. Eidenberger H (2003) New perspective on visual information retrieval. In: SPIE IS&T electronic imaging conference (storage and retrieval methods and applications for multimedia). SPIE proceedings vol 5307, San Jose, USA, pp 133–144

  15. Meyer zu Eissen S, Stein B, Potthast M (2005) The suffix tree document model revisited. In: Proceedings of the 5th international conference on knowledge management, I-KNOW, Graz, Austria, pp 598–603

  16. Eysenck MW (2004) Psychology: an international perspective. Psychology Press, chap Research Methods: Appendices

  17. Fonseca M (2004) Sketch-based retrieval in large sets of drawings. PhD thesis, Universidade Tecnica de Lisboa, graph Matching survey and spatial access methods survey

  18. Frankfort-Nachmias C, Nachmias D (1992) Research methods in the social sciences. St. Martin’s Press

  19. Hammiche S, Benbernou S, Hacid MS, Vakali A (2004) Semantic retrieval of multimedia data. In: MMDB ’04: proceedings of the 2nd ACM international workshop on multimedia databases, ACM Press, New York, NY, USA, pp 36–44

    Chapter  Google Scholar 

  20. Hunter J (2001) Adding multimedia to the semantic web - building an mpeg-7 ontology. In: First semantic web working symposium (SWWS), Stanford, USA, pp 261–281

  21. Kosch H (2003) Distributed multimedia database technologies. CRC Press

  22. Lux M (2008) Revisiting the vector retrieval model in context of the mpeg-7 semantic description scheme. In: Proccedings of the WIAMIS 2008, IEEE, Klagenfurt

  23. Lux M (2009) Caliph & Emir: Mpeg-7 photo annotation and retrieval. In: Proceedings of the seventeen ACM international conference on Multimedia, ACM, New York, NY, USA, MM ’09, pp 925–926, doi:10.1145/1631272.1631456, URL:http://doi.acm.org/10.1145/1631272.1631456

  24. Robertson S, Zaragoza H, Taylor M (2004) Simple bm25 extension to multiple weighted fields. In: CIKM ’04: proceedings of the thirteenth ACM international conference on Information and knowledge management, ACM Press, New York, NY, USA, pp 42–49

    Chapter  Google Scholar 

  25. Robertson SE, Walker S (1994) Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In: SIGIR ’94: proceedings of the 17th annual international ACM SIGIR conference on research and development in information retrieval, Springer-Verlag New York, Inc., New York, NY, USA, pp 232–241

    Google Scholar 

  26. Salembier P, Smith JR (2001) Mpeg-7 multimedia description schemes. IEEE Trans Circuits Syst Video Technol 11(6):748–759

    Article  Google Scholar 

  27. Santini S, Jain R (1998) Beyond query by example. In: 1998 IEEE second workshop on multimedia signal processing, IEEE, Redondo Beach, CA, USA, pp 3–8

    Chapter  Google Scholar 

  28. Shasha D, Wang JTL, Giugno R (2002) Algorithmics and applications of tree and graph searching. In: PODS ’02: Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, ACM Press, pp 39–52

  29. Shokoufandeh A, Dickinson SJ, Siddiqi K, Zucker S (1999) Indexing using a spectral encoding of topological structure. In: Conference on computer vision and pattern recognition, vol 2. IEEE Computer Society, USA, pp 491–497

    Google Scholar 

  30. Smeulders A, Worring M, Santini S, Gupta A, Jain R (2000) Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22(12):1349–1380

    Article  Google Scholar 

  31. Sowa JF (1979) Semantics of conceptual graphs. In: Proceedings of the 17th annual meeting on association for computational linguistics, association for computational linguistics, Morristown, NJ, USA, pp 39–44

  32. Tous R, Delgado J (2006) A vector space model for semantic similarity calculation and owl ontology alignment. In: Bressan S, Küng J, Wagner R (eds) Database and expert systems applications. Lecture notes in computer science, vol 4080. Springer Berlin / Heidelberg, pp 307–316

    Chapter  Google Scholar 

  33. Troncy R, Bailer W, Hausenblas M, Hofmair P, Schlatte R (2006) Enabling multimedia metadata interoperability by defining formal semantics of mpeg-7 profiles. In: Avrithis Y, Kompatsiaris Y, Staab S, O’Connor N (eds) Semantic multimedia. Lecture Notes in Computer Science, vol 4306. Springer Berlin / Heidelberg, pp 41–55

    Chapter  Google Scholar 

  34. Tsinaraki C, Fatourou E, Christodoulakis S (2003) An ontology-driven framework for the management of semantic metadata describing audiovisual information. In: 15th conference on advanced information systems engineering CAiSE 2003. LNCS, Springer

  35. Valiente G (2002) Algorithms on trees and graphs. Springer, Berlin, Germany

    MATH  Google Scholar 

  36. Yoon K, Doeller M, Gruhne M, Tous R, Sano M, Choi M, Lim TB, Lee JJ, Seo HC (2008) ISO/IEC IS 153938-12:2008: Information technology—multimedia content description interface — Part 12: query format. International Organization for Standardization, Geneva, Switzerland

  37. Zhong J, Zhu H, Li J, Yu Y (2002) Conceptual graph matching for semantic search. In: ICCS ’02: proceedings of the 10th international conference on conceptual structures, Springer-Verlag, London, UK, pp 92–196

    Google Scholar 

Download references

Acknowledgements

An implementation of the presented metric, the evaluation and a sample search engine based on a path index is available in context of the open source project Caliph & Emir (download and additional information on http://www.semanticmetadata.net).

Part of this work has been funded by the Know-Center. The Know-Center is a Competence Center funded within the Austrian Competence Center program K plus under the auspices of the Austrian Ministry of Transport, Innovation and Technology (www.kplus.at), by the state of Styria and by the City of Graz.

Author information

Authors and Affiliations

  1. Institute for Information Technology, Klagenfurt University, Klagenfurt, Austria

    Mathias Lux

Authors
  1. Mathias Lux
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mathias Lux.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lux, M. How to search in MPEG-7 based semantic descriptions: an evaluation of metrics. Multimed Tools Appl 59, 673–690 (2012). https://doi.org/10.1007/s11042-011-0756-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-011-0756-7

Keywords

Search

Navigation