Skip to main content
SpringerLink
Log in

Investigating fuzzy DLs-based reasoning in semantic image analysis

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Recent advances in semantic image analysis have brought forth generic methodologies to support concept learning at large scale. The attained performance however is highly variable, reflecting effects related to similarities and variations in the visual manifestations of semantically distinct concepts, much as to the limitations issuing from considering semantics solely in the form of perceptual representations. Aiming to enhance performance and improve robustness, we investigate a fuzzy DLs-based reasoning framework, which enables the integration of scene and object classifications into a semantically consistent interpretation by capturing and utilising the underlying semantic associations. Evaluation with two sets of input classifiers, configured so as to vary with respect to the wealth of concepts’ interrelations, outlines the potential of the proposed approach in the presence of semantically rich associations, while delineating the issues and challenges involved.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. http://www.w3.org/2001/sw/

  2. Intuitively a fuzzy assertion of the form a:C ≥ n means that the membership degree of the individual a to the concept C is at least equal to n.

  3. http://www.image.ece.ntua.gr/~nsimou

  4. http://faure.isti.cnr.it/~straccia/software/fuzzyDL/fuzzyDL.html

References

  1. Assfalg J, Bertini M, Colombo C, Bimbo AD (2002) Semantic annotation of sports videos. IEEE Multimed 9(2):52–60

    Article  Google Scholar 

  2. Baader F, Calvanese D, McGuinness DL, Nardi D, Patel-Schneider PF (2003) e.: the description logic handbook: theory, implementation, and applications. In: Description logic handbook. Cambridge University Press, Cambridge

    Google Scholar 

  3. Bagdanov A, Bertini M, DelBimbo A, Serra G, Torniai C (2007) Semantic annotation and retrieval of video events using multimedia ontologies. In: Proc. IEEE international conference on semantic computing (ICSC), Irvine, CA, USA

  4. Bechhofer S, van Harmelen F, Hendler J, Horrocks I, McGuinness D, Patel-Schneider P, Stein L (2004) OWL web ontology language reference, W3C Recommendation 10 February. http://www.w3.org/TR/owl-ref/

  5. Bell D, Qi G, Liu W (2007) Approaches to inconsistency handling in description-logic based ontologies. In: Proc. OTM workshops, Vilamoura, Portugal, pp 1303–1311

  6. Bobillo F, Straccia U (2008) Fuzzydl: an expressive fuzzy description logic reasoner. In: Proc. international conference on fuzzy systems (FUZZ). IEEE Computer Society, Hong Kong, pp 923–930

  7. Brickley D, Guha RV (2004) RDF Vocabulary description language 1.0: RDF schema, W3C Recommendation 10 February. http://www.w3.org/TR/rdf-schema/

  8. Crevier D, Lepage R (1997) Knowledge-based image understanding systems: a survey. Comput Vis Image Underst 67:161–185

    Article  Google Scholar 

  9. Dubois D, Prade H (2001) Possibility theory, probability theory and multiple-valued logics: a clarification. Ann Math Artif Intell 32(1-4):35–66

    Article  MathSciNet  Google Scholar 

  10. Elfers C, Herzog O, Miene A, Wagner T (2008) Qualitative abstraction and inherent uncertainty in scene recognition. In: Cohn AG, Hogg DC, Möller R, Neumann B (eds) Logic and B probabilty for scene interpretation, Dagstuhl Seminar Proceedings, Wadern

  11. Espinosa S, Kaya A, Melzer S, Möller R, Wessel M (2007) Multimedia interpretation as abduction. In: Proc. international workshop on description logics (DL), Brixen-Bressanone, Italy

  12. Haase P, Qi G (2007) An analysis of approaches to resolving inconsistencies in dl-based ontologies. In: Proc. international workshop on ontology dynamics (IWOD), Innsbruck, Austria, pp 97–109

  13. Haase P, van Harmelen F, Huang Z, Stuckenschmidt H, Sure Y (2005) A framework for handling inconsistency in changing ontologies. In: Proc. of international semantic web conference (ISWC), Galway, Ireland, pp 353–367

  14. Hanjalic A, Lienhart R, Ma W, Smith J (2008) The holy grail of multimedia information retrieval: so close or yet so far away. IEEE Proceedings (Special Issue on Multimedia Information Retrieval) 96(4):541–547

    Google Scholar 

  15. Hauptmann A, Yan R, Lin W (2007) How many high-level concepts will fill the semantic gap in news video retrieval? In: Proc. 6th ACM international conference on image and video retrieval (CIVR), Amsterdam, The Netherlands, pp 627–634

  16. Hunter J, Drennan SL (2004) Realizing the hydrogen economy through semantic web technologies. IEEE Intelligent Systems Journal—Special Issue on eScience 19:40–47

    Article  Google Scholar 

  17. Kalyanpur A, Parsia B, Sirin E, Grau BC (2006) Repairing unsatisfiable concepts in owl ontologies. In: Proc. of European semantic web conference (ESWC), Budva, Montenegro, pp 170–184

  18. Klir G, Yuan B (1995) Fuzzy sets and fuzzy logic: theory and applications. Prentice-Hall, Englewood Cliffs

    MATH  Google Scholar 

  19. Lam J, Sleeman D, Pan J, Vasconcelos W (2008) A fine-grained approach to resolving unsatisfiable ontologies. J Data Semantics 10:62–95

    Article  Google Scholar 

  20. LeBorgne H, Guérin-Dugué A, O’Connor N (2007) Learning midlevel image features for natural scene and texture classification. IEEE Trans Circuits Syst Video Technol 17(3):286–297

    Article  Google Scholar 

  21. Little S, Hunter J (2004) Rules-by-example—a novel approach to semantic indexing and querying of images. In: International semantic web conference (ISWC), Hiroshima, Japan, pp 534–548

  22. Moller R, Neumann B, Wessel M (1999) Towards computer vision with description logics: some recent progress. In: Proceedings integration of speech and image understanding, Corfu, Greece, pp 101–115

  23. Moosmann F, Triggs B, Jurie F (2006) Randomized clustering forests for building fast and discriminative visual vocabularies. In: Neural information processing systems (NIPS)

  24. Mylonas P, Simou N, Tzouvaras V, Avrithis Y (2007) Towards semantic multimedia indexing by classification and reasoning on textual metadata. Knowledge acquisition from multimedia content workshop, Genova, Italy

  25. Naphade M, Huang T (2001) A probabilistic framework for semantic video indexing, filtering, and retrieval. IEEE Trans Multimedia 3(1):141–151

    Article  Google Scholar 

  26. Naphade M, Huang T (2002) Extracting semantics from audio-visual content: the final frontier in multimedia retrieval. IEEE Trans Neural Netw 13(4):793–810

    Article  Google Scholar 

  27. Naphade M, Kennedy L, Kender J, Chang SF, Smith J, Over P, Hauptmann A (2005) A light scale concept ontology for multimedia understanding for trecvid 2005. In: RC23612 W0505-104, computer science, IBM Research Report

  28. Natsev A, Jiang W, Merler M, Smith J, Tesic J, Xie L Yan R (2008) IBM Research TRECVID-2008 video retrieval system. In: Proc. TREC Video Retrieval Workshop, Gaithersburg

  29. Neumann B (2008) Bayesian compositional hierarchies—a probabilistic structure for scene interpretation. In: Dagstuhl seminar proceedings

  30. Neumann B, Moller R (2004) On scene interpretation with description logics (FBI-B-257/04)

  31. Neumann B, Möller R (2007) On scene interpretation with description logics. Image Vis Comput (Special Issue on Cognitive Vision) 26:82–101

    Google Scholar 

  32. Niemann H, Sagerer G, Schröder S, Kummert F (1990) Ernest: a semantic network system for pattern understanding. IEEE Trans Pattern Anal Mach Intell 12(9):883–905

    Article  Google Scholar 

  33. Papadopoulos G, Mylonas P, Mezaris V, Avrithis Y, Kompatsiaris I (2006) Knowledge-assisted image analysis based on context and spatial optimization. Int J Semantic Web Inf Syst 2(3):17–36

    Google Scholar 

  34. Petridis K, Bloehdorn S, Saathoff C, Simou N, Dasiopoulou S, Tzouvaras V, Handschuh S, Avrithis Y, Kompatsiaris I, Staab S (2006) Knowledge representation and semantic annotation of multimedia content. IEE Proc Vis Image Signal Process (Special issue on Knowledge-Based Digital Media Processing) 153:255–262

    Google Scholar 

  35. Rao A, Jain R (1988) Knowledge representation and control in computer vision systems. IEEE Expert 3:64–79

    Article  Google Scholar 

  36. Reiter R, Mackworth A (1990) A logical framework for depiction and image interpretation. Artif Intell 41:125–155

    Article  MathSciNet  Google Scholar 

  37. Richardson M, Domingos P (2006) Markov logic networks. Mach Learn 62(1–2):107–136

    Article  Google Scholar 

  38. Schober JP, Hermes T, Herzog O (2004) Content-based image retrieval by ontology-based object recognition. In: Proc. KI-2004 workshop on applications of description logics (ADL), Ulm, Germany

  39. Simou N, Athanasiadis T, Tzouvaras V, Kollias S (2007) Multimedia reasoning with f-shin. In: 2nd international workshop on semantic media adaptation and personalization, London, UK

  40. Smeaton A, Over P, Kraaij W (2006) Evaluation campaigns and trecvid. In: MIR ’06: proceedings of the 8th ACM international workshop on multimedia information retrieval. ACM, New York, pp 321–330

    Chapter  Google Scholar 

  41. Smeulders AWM, Worring M, Santini S, Gupta A, Jain R (2000) Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22(12):1349–1380

    Article  Google Scholar 

  42. Snoek C, Worring M, van Gemert J, Geusebroek J, Smeulders A (2006) The challenge problem for automated detection of 101 semantic concepts in multimedia. In: Proc. 14th ACM international conference on multimedia, Santa Barbara, CA, USA, pp 421–430

  43. Snoek C, Huurnink B, Hollink L, de Rijke M, Schreiber G, Worring M (2007) Adding semantics to detectors for video retrieval. IEEE Trans Multimedia 9(5):975–986

    Article  Google Scholar 

  44. Snoek C, van de Sande K, deRooij O, Huurnink B, van Gemert J, Uijlings J, He J, Li X, Everts I, Nedovic V, van Liempt M, van Balen R, Yan F, Tahir M, Mikolajczyk K, Kittler J, de Rijke M, Geusebroek J, Gevers T, Worring M, Smeulders A, Koelma D (2008) The MediaMill TRECVID 2008 semantic video search engine. University of Amsterdam, Amsterdam

    Google Scholar 

  45. Stoilos G, Stamou G, Pan J (2006) Handling imprecise knowledge with fuzzy description logic. In: Proc. international workshop on description logics (DL), Lake District, UK

  46. Stoilos G, Stamou G, Pan J, Tzouvaras V, Horrocks I (2007) Reasoning with very expressive fuzzy description logics. J Artif Intell Res (JAIR) 30:273–320

    MATH  MathSciNet  Google Scholar 

  47. Stoilos G, Stamou G, Tzouvaras V, Pan J, Horrocks I (2005) The fuzzy description logic f-SHIN. In: International workshop on uncertainty reasoning for the semantic web (URSW), Galway, Ireland

  48. Straccia U (2001) Reasoning within fuzzy description logics. J Artif Intell Res (JAIR) 14:137–166

    MATH  MathSciNet  Google Scholar 

  49. Straccia U (2004) Transforming fuzzy description logics into classical description logics. In: Proc. European conference on logics in artificial intelligence (JELIA), Lisbon, Portugal, pp 385–399

  50. Straccia U (2006) A fuzzy description logic for the semantic web. In: Sanchez E (ed) Fuzzy logic and the semantic web, capturing intelligence. Elsevier, Amsterdam, pp 73–90

    Google Scholar 

  51. Town C, Sinclair D (2003) A self-referential perceptual inference framework for video interpretation. In: International confernce on computer vision systems (ICVS), Graz, Austria, pp 54–67

  52. Umberto S, Giulio V (2007) Dlmedia: an ontology mediated multimedia information retrieval system. In: Proc. international workshop on description logics (DL), Brixen-Bressanone, Italy

  53. Zadeh L (1965) Fuzzy sets. Inf Control 8(32):338–353

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was partially supported by the European Commission under contracts FP6-001765 aceMedia, FP6-507482 KnowledgeWeb and FP7-215453 WeKnowIt.

Author information

Authors and Affiliations

  1. Informatics and Telematics Institute, Centre for Research and Technology Hellas, Thessaloniki, Greece

    Stamatia Dasiopoulou, Ioannis Kompatsiaris & Michael G. Strintzis

Authors
  1. Stamatia Dasiopoulou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ioannis Kompatsiaris
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael G. Strintzis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stamatia Dasiopoulou.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Dasiopoulou, S., Kompatsiaris, I. & Strintzis, M.G. Investigating fuzzy DLs-based reasoning in semantic image analysis. Multimed Tools Appl 49, 167–194 (2010). https://doi.org/10.1007/s11042-009-0393-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-009-0393-6

Keywords

Search

Navigation