skip to main content
10.1145/2908446.2908471acmotherconferencesArticle/Chapter ViewAbstractPublication PagesinfosConference Proceedingsconference-collections
research-article

Collaborative Video Annotation Based on Ontological Themes, Temporal Duration and Pointing Regions

Published: 09 May 2016 Publication History

Abstract

The development of standards like MPEG-7, MPEG-21 and ID3 tags in MP3 have recognized the importance of adding descriptions to multimedia content for the purpose of better organization and retrieval. However, these standards are limited in scope and are only suitable for closed world multimedia content where a lot of effort is put in the production stage. On the contrary, video content on the Web is of arbitrary nature, captured and uploaded in a variety of formats, with the primary aim of quick and easy sharing. The advent of Web 2.0 has resulted in the wide availability of different video-sharing applications like YouTube and has made video as a major content on the Web. These web applications not only allow users to browse and search multimedia content but also add comments and annotations which provide an opportunity to harvest wisdom of the crowd. However, these annotations have not been exploited to their fullest potential for the purpose of searching and retrieval. Video searching, ranking and recommendations could become more efficient if these annotations are made machine-processable under the guidance of domain-level ontologies. Moreover, associating annotations with a specific region, temporal duration and/or a specific theme of a video results in faster retrieval of required video scene of clip. In this paper, we propose a collaborative video annotation system that is based on temporal duration and pointing regions inside a video and also utilizing ontological themes of the selected domain. For the proof-of-concept development and evaluation, a comprehensive sports ontology (Cricket in this case) has been designed. The proposed system performs well in the context of free-text and ontological annotations. It performs at a higher level when browsing and searching related themes, scenes and objects as well as summarizing related themes, scenes and objects.

References

[1]
Lella, A. 2014) comScore Releases March 2014 U.S. Online Video Rankings. comScore. http://www.comscore.com/Insights/Press-Releases?keywords=&tag=YouTube&country=&publishdate=l1y&searchBtn=GO. Accessed 25-12-2014
[2]
O'Hara, K. and Sellen, A. 1997. A comparison of reading paper and on-line documents. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems. ACM, pp 335--342
[3]
Haslhofer, B. Jochum, W. King, R. Sadilek, C. and Schellner K. 2009. The LEMO annotation framework: weaving multimedia annotations with the web. International Journal on Digital Libraries 10 (1):15--32
[4]
Simon, R. Jung, J. and Haslhofer, B. 2011. The YUMA media annotation framework. In: Research and Advanced Technology for Digital Libraries. Springer, pp 434--437
[5]
Kipp, M. 2001. ANVIL - A Generic Annotation Tool for Multimodal Dialogue. In Proceedings of the 7th European Conference on Speech Communication and Technology (Eurospeech)
[6]
Kipp M Spatiotemporal coding in anvil. In Proceedings of the 6th international conference on Language Resources and Evaluation (LREC-08), 2008.
[7]
Sloetjes, H. Russel, A. and Klassmann, A. 2008. ELAN: a free and open-source multimedia annotation tool. In Proceedings of the Proc. of INTERSPEECH. pp 4015--4016
[8]
Schallauer, P. Ober, S. and Neuschmied, H. 2008. Efficient semantic video annotation by object and shot re-detection. In Proceedings of the Posters and Demos Session, 2nd International Conference on Semantic and Digital Media Technologies (SAMT). Koblenz, Germany, 2008.
[9]
Chebotko, A. Deng, Y. Lu, S. Fotouhi, F. Aristar, A. Brugman, H. Klassmann, A. Sloetjes, H. Russel, A. and Wittenburg, P. 2004. OntoELAN.An ontology-based linguistic multimedia annotator. In proceeding of the IEEE 6th International Symposium on Multimedia Software Engineering. IEEE, pp 329--336
[10]
Hosack, B. 2010. VideoANT: Extending online video annotation beyond content delivery. Tech Trends 54 (3):45--49
[11]
Haslhofer, B. Momeni, E. Gay, M. and Simon, R. 2010. Augmenting Europeana content with linked data resources. In Proceedings of the 6th International Conference on Semantic Systems, ACM, p 40
[12]
Grassi, M. Morbidoni, C. and Nucci, M. 2011. Semantic web techniques application for video fragment annotation and management. In Analysis of Verbal and Nonverbal Communication and Enactment. The Processing Issues. Springer, pp 95--103
[13]
Morbidoni, C. Grassi, M. Nucci, M. Fonda, S. and Ledda, G. Introducing SemLib project: semantic web tools for digital libraries. n Proceedings of the International Workshop on Semantic Digital Archives-Sustainable Long-term Curation Perspectives of Cultural Heritage Held as Part of the 15th International Conference on Theory and Practice of Digital Libraries (TPDL), Berlin, 2011.
[14]
Lambert, D. and Yu, HQ. 2010. Linked Data based video annotation and browsing for distance learning. In SemHE, The second international workshop on Semantic Web Application in Higher Education. Southampton, UK.
[15]
Li, Y. Wald, M. Wills, G. Khoja. S. Millard, D. Kajaba, J. Singh, P. and Gilbert, L. 2011. Synote: development of a Web-based tool for synchronized annotations. New Review of Hypermedia and Multimedia 17 (3):295--312
[16]
Li, Y. Wald, M. Omitola, T. Shadbolt, N. and Wills, G. 2012. Synote: weaving media fragments and linked data.
[17]
van Lancker, W. Van Deursen, D. Mannens, E. and Van de Walle, R. 2012. Implementation strategies for efficient media fragment retrieval. Multimedia Tools and Applications 57 (2):243--267
[18]
YouTube Video Annotations. http://www.youtube.com/t/annotations_about. Accessed 7 January 2014
[19]
Kahan, J. Koivunen, M-R. 2001. Annotea: an open RDF infrastructure for shared Web annotations. In Proceedings of the 10th international conference on World Wide Web. ACM, pp 623--632
[20]
Schroeter, R. Hunter, J. Kosovic, D. 2003. Vannotea: A collaborative video indexing, annotation and discussion system for broadband networks. In Proceeding of the Knowledge capture. ACM Press (Association for Computing Machinery), pp 1--8
[21]
Haslhofer, B. Simon, R. Sanderson, R. and Van de Sompel, H. 2011. The open annotation collaboration (OAC) model. In Proceeding of the Multimedia on the Web (MMWeb), Workshop on, 2011. IEEE, pp 5--9
[22]
Kurz, T. Schaffert, S. and Burger, T. 2011. Lmf: A framework for linked media. In Proceeding of Multimedia on the Web (MMWeb), Workshop on, 2011. IEEE, pp 16--20
[23]
Mortensen, A. Gaddam, VR. Stensland, HK. Griwodz, C. Johansen, D. and Halvorsen, P. 2014. Automatic event extraction and video summaries from soccer games. In Proceedings of the 5th ACM Multimedia Systems Conference. pp 176--179
[24]
Qian, X. Liu, G. Wang, H. Li, Z. and Wang, Z. 2011. Soccer video event detection by fusing middle level visual semantics of an event clip. In Advances in Multimedia Information Processing-PCM 2010. Springer, pp 439--451
[25]
Rui, Y. Gupta, A. Acero, A.2000. Automatically extracting highlights for TV baseball programs. In Proceedings of the 8th ACM international conference on Multimedia. pp 105--115
[26]
Xu M, Maddage NC, Xu C, Kankanhalli M, Tian Q Creating audio keywords for event detection in soccer video. In: Multimedia and Expo, 2003. ICME'03. Proceedings. 2003 International Conference on, 2003. IEEE, pp II-281-284 vol. 282
[27]
Ekin A, Tekalp AM, Mehrotra R (2003) Automatic soccer video analysis and summarization. Image Processing, IEEE Transactions on 12 (7):796--807
[28]
Zhang, D. and Chang, S-F. 2002. Event detection in baseball video using superimposed caption recognition. In Proceedings of the tenth ACM international conference on Multimedia. ACM, pp 315--318
[29]
Assfalg, J. Bertini, M. Colombo, C. Del Bimbo, A. and Nunziati, W. 2003. Semantic annotation of soccer videos: automatic highlights identification. Computer Vision and Image Understanding 92 (2):285--305
[30]
Xu, C. Zhang, Y-F. Zhu, G. Rui, Y. Lu, H. and Huang, Q. 2008. Using webcast text for semantic event detection in broadcast sports video. IEEE, Multimedia Transactions on 10 (7):1342--1355
[31]
Kumar, YS. Gupta, SK. Kiran, BR. Ramakrishnan, K. and Bhattacharyya, C. 2011. Automatic summarization of broadcast cricket videos. In proceeding of theIEEE 15th International Symposium on Consumer Electronics (ISCE). IEEE, pp 222--225
[32]
Ouyang, J-q. Jin-tao, L. and Yong-dong, Z. 2004. Ontology based sports video annotation and summary. In Content Computing. Springer, pp 499--508
[33]
Xu, C. Wang, J. Lu, H. and Zhang, Y. 2008. A novel framework for semantic annotation and personalized retrieval of sports video. Multimedia, IEEE Transactions on 10 (3):421--436
[34]
Xu, M. Duan, L-Y. Xu, C-S. and Tian, Q. 2003. A fusion scheme of visual and auditory modalities for event detection in sports video. In Proceeding of the international conference on Acoustics, Speech, and Signal Processing (ICASSP). IEEE, pp III-189-192 vol. 183
[35]
Wan, K. and Xu, C. 2004. Efficient multimodal features for automatic soccer highlight generation. In Proceedings of the 17th International Conference on Pattern Recognition (ICPR). IEEE, pp 973--976
[36]
Patil, S. and Jadhav, D. 2012. Semantic information retrieval using ontology and sparql for cricket. International Journal of Advances in Engineering & Technology 4 (2):354--363
[37]
Kosamkar, P. 2012. Annotation Based Event Retrieval in Cricket Video. International Journal of Advances in Computing and Information Researches 1 (2):6--9.

Cited By

View all
  • (2023)A Taxonomy of Methods, Tools, and Approaches for Enabling Collaborative AnnotationProceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems10.1145/3638067.3638074(1-12)Online publication date: 16-Oct-2023
  • (2019)Improving Youtube video retrieval by integrating crowdsourced timed metadataJournal of Intelligent & Fuzzy Systems10.3233/JIFS-179333(1-15)Online publication date: 16-Jul-2019
  • (2018)On the Current State of Linked Open DataInternational Journal on Semantic Web & Information Systems10.4018/IJSWIS.201810010614:4(110-128)Online publication date: 1-Oct-2018
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
INFOS '16: Proceedings of the 10th International Conference on Informatics and Systems
May 2016
347 pages
ISBN:9781450340625
DOI:10.1145/2908446
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 May 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Ontologies
  2. annotations
  3. video sharing applications

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

INFOS '16

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1
  • Downloads (Last 6 weeks)1
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2023)A Taxonomy of Methods, Tools, and Approaches for Enabling Collaborative AnnotationProceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems10.1145/3638067.3638074(1-12)Online publication date: 16-Oct-2023
  • (2019)Improving Youtube video retrieval by integrating crowdsourced timed metadataJournal of Intelligent & Fuzzy Systems10.3233/JIFS-179333(1-15)Online publication date: 16-Jul-2019
  • (2018)On the Current State of Linked Open DataInternational Journal on Semantic Web & Information Systems10.4018/IJSWIS.201810010614:4(110-128)Online publication date: 1-Oct-2018
  • (2018)YouTube Timed Metadata Enrichment Using a Collaborative ApproachCryptology and Network Security10.1007/978-3-319-98678-4_15(131-141)Online publication date: 15-Aug-2018

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media