skip to main content
10.1145/1460676.1460678acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
keynote

Learning ontology rules for semantic video annotation

Published: 31 October 2008 Publication History

Abstract

Semantic video annotation using ontologies has received a large attention from the scientific community in the recent years. Ontologies are being regarded as an appropriate tool to bridge the semantic gap. In this paper we present an overview of the state-of-the-art of approaches and algorithms that exploit ontologies to perform semantic video annotation and present an approach to automatically learn rules describing high-level concepts. This approach exploits the domain knowledge embedded into an ontology to learn a set of rules for semantic video annotation. The proposed technique is an adaptation of the First Order Inductive Learner (FOIL) technique to the Semantic Web Rule Language (SWRL) standard: Experiments have been performed in two different video domains: i) the TRECVID 2005 broadcast news collection, to detect events related to airplanes, such as taxiing, flying, landing and taking off; ii) surveillance videos, to detect if a person enters or exits a specific area. The promising experimental performance demonstrates the effectiveness and flexibility of the proposed framework.

References

[1]
Dublin Core Metadata Initiative - http://dublincore.org/.
[2]
TV Anytime Forum - http://www.tv-anytime.org/.
[3]
R. Arndt, R. Troncy, S. Staab, L. Hardman, and M. Vacura. Comm: Designing a well-founded multimedia ontology for the web. In Proc. of Int'l Semantic Web Conference, 2007.
[4]
A. D. Bagdanov, A. Del Bimbo, F. Dini, and W. Nunziati. Improving the robustness of particle filter-based visual trackers using online parameter adaptation. In Proc. of IEEE Int'l Conference on Advanced Video and Signal Based Surveillance, 2007.
[5]
L. Bai, S. Lao, G. Jones, and A. F. Smeaton. Video semantic content analysis based on ontology. In Proc. of Int'l Machine Vision and Image Processing Conference, 2007.
[6]
M. Bertini, A. Del Bimbo, C. Torniai, R. Cucchiara, and C. Grana. Dynamic pictorial ontologies for video digital libraries annotation. In Proc. ACM Int'l Workshop on the Many Faces of Multimedia Semantics, 2007.
[7]
S. Bloehdorn, K. Petridis, C. Saathoff, N. Simou, V. Tzouvaras, Y. Avrithis, S. Handschuh, I. Kompatsiaris, S. Staab, and M. Strintzis. Semantic annotation of images and videos for multimedia analysis. In Proc. of European Semantic Web Conference, 2005.
[8]
S. Castano, S. Espinosa, A. Ferrara, V. Karkaletsis, A. Kaya, S. Melzer, R. Moller, S. Montanelli, and G. Petasis. Ontology dynamics with multimedia information: The boemie evolution methodology. In Proc. of Int'l Workshop on Ontology Dynamics, 2007.
[9]
S. Dasiopoulou, V. Mezaris, I. Kompatsiaris, V. K. Papastathis, and M. G. Strintzis. Knowledge-assisted semantic video object detection. IEEE Trans. on Circuits and Systems for Video Technology, 15(10):1210--1224, 2005.
[10]
S. Dasiopoulou, C. Saathoff, P. Mylonas, Y. Avrithis, Y. Kompatsiaris, S. Staab, and M. Strintzis. Semantic Multimedia and Ontologies Theory and Applications, chapter Introducing Context and Reasoning in Visual Content Analysis: An Ontology-Based Framework, pages 99--122. Springer, 2008.
[11]
A. Dorado, J. Calic, and E. Izquierdo. A rule-based video annotation system. Circuits and Systems for Video Technology, IEEE Transactions on, 14(5):622--633, May 2004.
[12]
S. Ebadollahi, L. Xie, S.-F. Chang, and J. Smith. Visual event detection using multi-dimensional concept dynamics. In Proc. of IEEE Int'l Conference on Multimedia & Expo, 2006.
[13]
S. Espinosa, A. Kaya, S. Melzer, R. Moller, and M. Wessel. Towards a media interpretation framework for the semantic web. In Proc. of Int'l Conference on Web Intelligence, 2007.
[14]
A. Francois, R. Nevatia, J. Hobbs, R. Bolles, and J. Smith. VERL: an ontology framework for representing and annotating video events. IEEE Multimedia, 12(4):76--86, Oct-Dec. 2005.
[15]
R. Garcia and O. Celma. Semantic integration and retrieval of multimedia metadata. In Proc. of the Knowledge Markup and Semantic Annotation Workshop, 2005.
[16]
A. Hauptmann, M. Chen, M.-Y.and Christel, W.-H. Lin, and J. Yang. A hybrid approach to improving semantic extraction of news video. In Proc. of IEEE Int'l Conference on Semantic Computing, 2007.
[17]
L. Hollink, S. Little, and J. Hunter. Evaluating the application of semantic inferencing rules to image annotation. In Proc. of Int'l Conference on Knowledge Capture, 2005.
[18]
L. Kennedy. Revision of LSCOM event/activity annotations, DTO challenge workshop on large scale concept ontology for multimedia. Advent technical report #221-2006-7, Columbia University, 2006.
[19]
M. Koskela, A. F. Smeaton, and J. Laaksonen. Measuring concept similarities in multimedia ontologies: Analysis and evaluation. IEEE Transactions on Multimedia, 9(5):912--922, August 2007.
[20]
L. Leslie, T.-S. Chua, and J. Ramesh. Annotation of paintings with high-level semantic concepts using transductive inference and ontology-based concept disambiguation. In Proc. ACM Multimedia, 2007.
[21]
K.-H. Liu, M.-F. Weng, C.-Y. Tseng, Y.-Y. Chuang, and M.-S. Chen. Association and temporal rule mining for post-filtering of semantic concept detection in video. Multimedia, IEEE Transactions on, 10(2):240--251, Feb. 2008.
[22]
N. Maillot and M. Thonnat. Ontology based complex object recognition. Image Vision Computing, 26(1):102--113, 2008.
[23]
M. Naphade, J. Smith, J. Tesic, S.-F. Chang, L. Kennedy, A. Hauptmann, and J. Curtis. Large-scale concept ontology for multimedia. IEEE Multimedia, 13(3):86--91, July-Sept. 2006
[24]
B. Neumann and R. Moeller. On scene interpretation with description logics. In Cognitive Vision Systems: Sampling the Spectrum of Approaches, LNCS, pages 247--278. Springer, 2006.
[25]
J. R. Quinlan. Learning logical definitions from relations. Machine Learning, 5(3):239--266, 1990.
[26]
M.-L. Shyu, Z. Xie, M. Chen, and S.-C. Chen. Video semantic event/concept detection using a subspace-based multimedia data mining framework. Multimedia, IEEE Transactions on, 10(2):252--259, Feb. 2008.
[27]
C. Snoek, B. Huurnink, L. Hollink, M. de Rijke, G. Schreiber, and M. Worring. Adding semantics to detectors for video retrieval. 9(5):975--986, Aug. 2007.
[28]
C. Snoek and M. Worring. Multimedia event-based video indexing multimedia event-based video indexing using time intervals. IEEE Transactions on Multimedia, 7(4):638--647, 2005.
[29]
C. Tsinaraki, P. Polydoros, F. Kazasis, and S. Christodoulakis. Ontology-based semantic indexing for MPEG-7 and TV-Anytime audiovisual content. Multimedia Tools and Applications, (26):299--325, Aug. 2005.
[30]
P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, 2001.
[31]
Z.-J. Zha, T. Mei, Z. Wang, and X.-S. Hua. Building a comprehensive ontology to refine video concept detection. In Proc. of ACM Int'l Workshop on Multimedia Information Retrieval, 2007.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MS '08: Proceedings of the 2nd ACM workshop on Multimedia semantics
October 2008
70 pages
ISBN:9781605583167
DOI:10.1145/1460676
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 October 2008

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. events detection
  2. learning rules
  3. ontology
  4. video retrieval

Qualifiers

  • Keynote

Conference

MM08
Sponsor:
MM08: ACM Multimedia Conference 2008
October 31, 2008
British Columbia, Vancouver, Canada

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1
  • Downloads (Last 6 weeks)1
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Ontology in Dance Domain—A SurveyJournal on Computing and Cultural Heritage 10.1145/369076718:1(1-32)Online publication date: 18-Oct-2024
  • (2020)On the use of semantic technologies for video analyticsJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-020-02021-yOnline publication date: 13-May-2020
  • (2018)Semantic-Based Video Retrieval SurveyJournal of Computer and Communications10.4236/jcc.2018.6800306:08(28-44)Online publication date: 2018
  • (2017)Extensible Hierarchical Method of Detecting Interactive Actions for Video UnderstandingETRI Journal10.4218/etrij.17.0116.005439:4(502-513)Online publication date: 11-Aug-2017
  • (2017)OVIS: ontology video surveillance indexing and retrieval systemInternational Journal of Multimedia Information Retrieval10.1007/s13735-017-0133-z6:4(295-316)Online publication date: 18-Sep-2017
  • (2017)BalOnSe: Temporal Aspects of Dance Movement and Its Ontological RepresentationThe Semantic Web10.1007/978-3-319-58451-5_4(49-64)Online publication date: 7-May-2017
  • (2015)Applications Exploiting Multimedia SemanticsMultimedia Ontology10.1201/b18639-16(177-200)Online publication date: 26-Jun-2015
  • (2015)Events Detection Using a Video-Surveillance Ontology and a Rule-Based ApproachComputer Vision - ECCV 2014 Workshops10.1007/978-3-319-16181-5_21(299-308)Online publication date: 20-Mar-2015
  • (2014)An ontology based approach for inferring multiple object events in surveillance domain2014 Science and Information Conference10.1109/SAI.2014.6918219(404-409)Online publication date: Aug-2014
  • (2013)Towards Ontological Cognitive SystemTopics in Medical Image Processing and Computational Vision10.1007/978-94-007-0726-9_5(87-99)Online publication date: 28-Mar-2013
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media