Abstract
A video retrieval system user hopes to find relevant information when the proposed queries are ambiguous. The retrieval process based on detecting concepts remains ineffective in such a situation. Potential relationships between concepts have been shown as a valuable knowledge resource that can enhance the retrieval effectiveness, even for ambiguous queries. Recent researches in multimedia retrieval have focused on ontology modeling as a common framework to manage knowledge. Handling these ontologies has to cope with issues related to generic knowledge management and processing scalability. Considering these issues, we suggest a context-based fuzzy ontology framework for video content analysis and indexing. In this paper, we focused on the way in which we modeled our fuzzy ontology: First, we populate automatically the generated ontology by gathering various available video annotation datasets. Then, the ontology content was used to infer enhanced video semantic interpretation. Finally, considering user feedback, the content of the ontology was improved. Experimental results showed that our approach achieves the goal of scalability while at the same time allowing better video content semantic interpretation.


Similar content being viewed by others
References
(2006). LSCOM Lexicon Definitions Version, Annotations, Version 1.0., Tech. rep. Columbia University
Adami N, Bugatti A, Leonardi R, Migliorati P (2001) Low level processing of audio and video information for extracting the semantics of content. In: 2001 IEEE Fourth Workshop on Multimedia Signal Processing, pp 607–612
Ayache S (2007) Indexation de documents vidéos par concepts par fusion de caractristiques audio, image et texte, Ph.D. thesis, Institut National Polytechnique de Grenoble
Ayache S (2008) Video Corpus Annotation using Active Learning. In: European Conference on Information Retrieval (ECIR). Glasgow, Scotland, pp 187–198
Baader F, Calvanese D, McGuinness D L, Nardi D, Patel-Schneider PF (eds) (2003) The description logic handbook: theory, implementation, and applications. Cambridge University Press, New York
Baghdadi S, Gravier G, Demarty C, Gros P (2008) Structure learning in a bayesian network-based video indexing framework. In: 2008 IEEE International Conference on Multimedia and Expo, pp. 677–680
Bannour H, Hudelot C (2013) Building and using fuzzy multimedia ontologies for semantic image annotation. Multimed Tools Appl:1–35
Benitez A, Chang SF (2003) Image classification using multimedia knowledge networks. In: 2003. ICIP 2003. Proceedings. 2003 International Conference on Image Processing, vol. 3, pp. III–613–16 vol.2
Bobillo F, Delgado M, Gmez-Romero J, Straccia U (2012) Joining gödel and zadeh fuzzy logics in fuzzy description logics, vol 20
Bosko B (1990) Fuzziness vs. probability. Int J Gen Syst 17(2-3):211–240
Brilhault A (2009) Indexation et recherche par le contenu de documents vidéos, Joseph Fourier University
Calegari S, Ciucci D (2007) Fuzzy ontology, fuzzy description logics and fuzzy-owl. In: Proceedings of the 7th international workshop on Fuzzy Logic and Applications: Applications of Fuzzy Sets Theory, WILF ’07. Springer-Verlag, Berlin, Heidelberg, pp 118–126
Chattopadhyay C, Maurya A (2013) Genre-specific modeling of visual features for efficient content based video shot classification and retrieval. Int J Multimed Inf Retr 2(4):289–297. doi:10.1007/s13735-013-0034-8
Cheng Y, Xiong Y (2012) Research on model of ontology-based semantic information retrieval. In: Jin D, Lin S (eds) Advances in Multimedia, Software Engineering and Computing Vol.1, vol 128. Springer , Berlin Heidelberg, pp 271–276
Dasiopoulou S, Giannakidou E, Litos G, Malasioti P, Kompatsiaris Y (2011) A survey of semantic image and video annotation tools. In: Paliouras G, Spyropoulos C , Tsatsaronis G (eds) Knowledge-Driven Multimedia Information Extraction and Ontology Evolution, vol 6050. Springer, Berlin Heidelberg, pp 196–239
Dean J (2009) Challenges in building large-scale information retrieval systems: invited talk. In: Proceedings of the Second ACM International Conference on Web Search and Data Mining, WSDM ’09, New York, pp 1–1
DeMenthon D, Megret R (2002) Spatio-Temporal Segmentation of Video by Hierarchical Mean Shift Analysis. Tech. Rep. LAMP-TR-090, CAR-TR-978, CS-TR-4388, UMIACS-TR-2002-68, University of Maryland, College Park
Dentler K, Cornet R, Ten Teije A, De Keizer N (2011) Comparison of reasoners for large ontologies in the owl 2 el profile. Semant. web 2(2):71–87
Du Y, Chen F, Xu W, Zhang W (2006) Interacting activity recognition using hierarchical durational-state dynamic bayesian network. In: Zhuang Y, Yang S Q, Rui Y, He Q (eds) Advances in Multimedia Information Processing - PCM 2006, vol 4261. Springer , Berlin Heidelberg, pp 185–192
Egozi O, Markovitch S, Gabrilovich E (2011) Concept-based information retrieval using explicit semantic analysis. ACM Trans Inf Syst 29(2):8:1–8:34
Elleuch N, Ben Ammar A, Alimi A M (2010) A generic system for semantic video indexing by visual concept. In: 2010 5th International Symposium on I/V Communications and Mobile Network (ISVC)
Elleuch N, Zarka M, Ben Ammar A, Alimi MA (2011) A fuzzy ontology: based framework for reasoning in visual video content analysis and indexing. In: Proceedings of the Eleventh International Workshop on Multimedia Data Mining, MDMKDD ’11. New York, pp 1–1
Elleuch N, Zarka M, Feki I, Ben Ammar A, Alimi MA (2010) Regimvid at trecvid 2010: Semantic indexing, TRECVID. 2010
Faria C, Girardi R (2011) An information extraction process for semi-automatic ontology population. In: Corchado E, Snel V, Sedano J , Hassanien A , Calvo J , lzak D (eds) Soft Computing Models in Industrial and Environmental Applications, 6th International Conference SOCO 2011, Advances in Intelligent and Soft Computing, vol 87, pp 319–328. Springer , Berlin Heidelberg
Fellbaum C (2010) Wordnet. In: Poli R, Healy M, Kameas A (eds) Theory and Applications of Ontology: Computer Applications. Springer , Netherlands, pp 231–243
Fernndez-López M (1999) Overview of methodologies for building ontologies. In: Proceedings of the IJCAI-99 Workshop on Ontologies and Problem Solving Methods (KRR5) Stockholm, Sweden, August 2, 1999
Fu G, Jones C, Abdelmoty A (2005) Ontology-based spatial query expansion in information retrieval. In: Meersman R, Tari Z (eds) On the Move to Meaningful Internet Systems 2005: CoopIS, DOA, and ODBASE, vol 3761. Springer, Berlin Heidelberg, pp 1466–1482
Gargouri F, Jaziri W (2010) Ontology Theory, Management and Design: Advanced Tools and Models. Premier Reference Source. Information Science Reference
Horrocks I (2012) Semantics ; scalability: Journal of Zhejiang University - Science C 13(4) 241–244
Huang YF, Wang SH (2012) Movie genre classification using svm with audio and video features. In: Huang R, Ghorbani A, Pasi G, Yamaguchi T, Yen N, Jin B (eds) Active Media Technology, vol 7669. Springer , Berlin Heidelberg, pp 1–10
Jiang Y G, Wang J, Chang S F, Ngo C W (2009) Domain adaptive semantic diffusion for large scale context-based video annotation. In: IEEE 12th International Conference on Computer Vision, pp 1420 –1427
Kara S (2010) An ontology-absed retrieval system using semantic indexing, Ph.D. thesis, Middle East Technical University
Ksentini N, Zarka M, Ben Ammar A, Alimi MA (2012) Toward an assisted context based collaborative annotation. In: 10th International Workshop on Content-Based Multimedia Indexing (CBMI), 2012, pp 1 –6
Ksibi A, Ben Ammar A, Ben Amar C (2014) Adaptive diversification for tag-based social image retrieval. IJMIR 3(1):29–39
Ksibi A, Dammak M, Ben Ammar A, Mejdoub M, Ben Amar C (2012) Flickr-based semantic context to refine automatic photo annotation. In: Image Processing Theory, Tools and Applications (IPTA), 2012 3rd International Conference on, pp 377–382
Kumar S, Rana RK, Singh P (2012) Ontology based semantic indexing approach for information retrieval system, vol 49, pp 14–18. Published by Foundation of Computer Science, New York, USA
Leite M, Ricarte I (2008) A framework for information retrieval based on fuzzy relations and multiple ontologies. In: Geffner H, Prada R, Machado Alexandre I, David N (eds) Advances in Artificial Intelligence IBERAMIA 2008, vol 5290. Springer , Berlin Heidelberg, pp 292–301
Li Z, Ramani K (2007) Ontology-based design information extraction and retrieval. AI EDAM 21:137–154
Mukesh R, Penchala S, Ingale A (2013) Ontology based zone indexing using information retrieval systems. In: Unnikrishnan S, Surve S , Bhoir D (eds) Advances in Computing, Communication, and Control, vol 361. Springer , Berlin Heidelberg, pp 181–186
Muneesawang P, Zhang N, Guan L (2014) Scalable video genre classification and event detection. In: Multimedia Database Retrieval, Multimedia Systems and Applications, pp 247–278. Springer International Publishing
Mustafa J, Khan S, Latif K (2008) Ontology based semantic information retrieval. In: 2008 IS ’08. 4th International IEEE Conference Intelligent Systems, vol 3, pp 22–14–22–19
Mylonas P, Athanasiadis T, Wallace M, Avrithis Y, Kollias S (2008) Semantic representation of multimedia content: Knowledge representation and semantic indexing. Multimed Tools Appl 39(3):293–327
Mylonas P, Spyrou E, Avrithis Y, Kollias S (2009) Using visual context and region semantics for high-level concept detection. Multimed, IEEE Trans on 11(2):229–243
Nguyen C T (2010) Bridging semantic gaps in information retrieval: Context-based approaches. In: VLDB doctoral workshop, Singapore 2010
Nikolopoulos S, Papadopoulos G, Kompatsiaris I, Patras I (2009) An evidence-driven probabilistic inference framework for semantic image understanding. In: Perner P (ed) Machine Learning and Data Mining in Pattern Recognition, vol 5632. Springer , Berlin Heidelberg, pp 525–539
Nikolopoulos S, Papadopoulos G T, Kompatsiaris I, Patras I (2011) Evidence-driven image interpretation by combining implicit and explicit knowledge in a bayesian network, IEEE Transactions on Systems, Man, and Cybernetics, Part B 41(5),1366–1381
Noy NF, Mcguinness D L (2001) Ontology development 101: A guide to creating your first ontology. Tech. Rep. KSL-01-05, Stanford Knowledge Systems Laboratory
Over P, Awad G, Michel M, Fiscus J, Sanders G, Kraaij W, Smeaton AF, Quenot G (2013) Trecvid 2013 – an overview of the goals, tasks, data, evaluation mechanisms and metrics. In: Proceedings of TRECVID 2013. NIST USA
Paliouras G, Spyropoulos C, Tsatsaronis G (2011) Bootstrapping ontology evolution with multimedia information extraction. In: Paliouras G , Spyropoulos C , Tsatsaronis G (eds) Knowledge-Driven Multimedia Information Extraction and Ontology Evolution, vol 6050 . Springer, Berlin Heidelberg, pp 1–17
Paliouras G, Spyropoulos CD, Tsatsaronis G (2011) Bootstrapping ontology evolution with multimedia information extraction. Lect Notes in Comput Sci 6050
Paliouras G, Spyropoulos C D, Tsatsaronis G (2011) Knowledge-Driven Multimedia Information Extraction and Ontology Evolution - Bridging the Semantic Gap, vol 6050. Springer
Park S, Aggarwal J (2004) A hierarchical bayesian network for event recognition of human actions and interactions. Multimedia Systems 10(2):164–179
Perpetual Coutinho F, Asnani K, Amos Caeiro D (2012) Context based information retrieval. International Journal of Advanced Research in Computer Science and Electronics Engineering (IJARCSEE) 1(7)
Petasis G, Karkaletsis V, Paliouras G, Krithara A, Zavitsanos E (2011) Ontology population and enrichment: State of the art
Petersohn C (2004) Fraunhofer hhi at trecvid 2004: Shot boundary detection system. In: TREC Video Retrieval Evaluation Online Proceedings, TRECVID
Petridis K, Bloehdorn S, Saathoff C, Simou N, Dasiopoulou S, Tzouvaras V, Handschuh S, Avrithis Y, Kompatsiaris Y, Staab S (2006) Knowledge representation and semantic annotation of multimedia content. Vision, Image and Signal Processing. IEE Proceedings - 153(3):255–262
Rodrguez-Garca M, Valencia-Garca R , Garca-Snchez F (2012) An ontology evolution-based framework for semantic information retrieval. In: Herrero P, Panetto H, Meersman R, Dillon T (eds) On the Move to Meaningful Internet Systems: OTM 2012 Workshops, vol 7567 . Springer , Berlin Heidelberg, pp 163–172
Romero AA, Grau BC, Horrocks I, Jiménez-Ruiz E (2013) More: a modular owl reasoner for ontology classification. In: Bail S , Glimm B, Gonçalves R S, Jiménez-Ruiz E, Kazakov Y , Matentzoglu N, Parsia B (eds) ORE, CEUR Workshop Proceedings, vol 1015, pp 61–67. CEUR-WS.org
Rozilawati binti D, Masaki A (2011) Ontology based approach for classifying biomedical text abstracts. International Journal of Data Engineering 2(1)
Sanjaa B, Tsoozol P (2007) Fuzzy and probability. In: Strategic Technology, 2007. IFOST 2007. International Forum on, pp. 141–143
Sari RF, Ayuningtyas N (2010) Implementation of web ontology and semantic application for electronic journal citation system. Journal Of Emerging Technologies in Web Intelligence 2:34–41
Simou N, Kollias S (2007) Fire: A fuzzy reasoning engine for impecise knowledge. K-Space PhD Students Workshop, Berlin, Germany, 14 September 2007
Smeaton AF, Over P, Kraaij W (2006) Evaluation campaigns and trecvid. In: Proceedings of the 8th ACM international workshop on Multimedia information retrieval, MIR ’06, pp 321–330, ACM, New York, NY, USA
Snoek CGM, Worring M (2009) Concept-based video retrieval. Foundations and Trends in Information Retrieval 2(4):215–322
Staab S, Studer R (2009) Handbook on Ontologies, 2nd edn. Springer Publishing Company, Incorporated
Stoilos G, Stamou GB, Tzouvaras V, Pan JZ, Horrocks I (2005) The fuzzy description logic f-shin. International Workshop on Uncertainty Reasoning For the Semantic Web (2005)
Thomee B, Popescu A (2012) Overview of the imageclef 2012 flickr photo annotation and retrieval task. In: Forner P , Karlgren J, Womser-Hacker C (eds) CLEF (Online Working Notes/Labs/Workshop)
Vallet D, Castells P, Fernandez M, Mylonas P, Avrithis Y (2007) Personalized content retrieval in context using ontological knowledge. Circuits and Systems for Video Technology. IEEE Transactions on 17(3):336–346
Wu F, Wu G, Fu X (2008) Design and implementation of ontology-based query expansion for information retrieval. In: Xu L, Tjoa A, Chaudhry S (eds) Research and Practical Issues of Enterprise Information Systems II, vol 254, pp 293–298. Springer US,
Wu J, Worring M (2012) Efficient genre-specific semantic video indexing. Multimedia, IEEE Transactions on 14(2 ):291–302 . doi:10.1109/TMM.2011.2174969
Zadeh L (2014) Fuzzy set theory and probability theory: What is the relationship? In: Lovric M (ed) International Encyclopedia of Statistical Science. Springer , Berlin Heidelberg, pp 563–566
Zarka M, Ben Ammar A, Alimi M A (2011) Multimodale fuzzy fusion for semantic video indexing. In: IEEE Symposium Series in Computational Intelligence 2011 - CIMSIVP
Zhai J, Li M, Li J (2012) Semantic information retrieval based on rdf and fuzzy ontology for university scientific research management. In: Luo J (ed) Affective Computing and Intelligent Interaction, vol 137. Springer , Berlin Heidelberg, pp 661–668
Acknowledgment
The authors would like to acknowledge the financial support of this work by grants from General Direction of Scientific Research (DGRST), Tunisia, under the ARUB program.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zarka, M., Ben Ammar, A. & Alimi, A.M. Fuzzy reasoning framework to improve semantic video interpretation. Multimed Tools Appl 75, 5719–5750 (2016). https://doi.org/10.1007/s11042-015-2537-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-015-2537-1