Abstract
Videos represent one of the most frequently used forms of multimedia applications. In addition to watching videos, people control slider bars of video players to find specific scenes and want detailed information on certain objects in scenes. However, it is difficult to support user interactions in current video formats because of a lack of metadata for facilitating such interactions. This paper proposes a text-based semantic video annotation system for interactive cooking videos to facilitate user interactions. The proposed annotation process includes three parts: the synchronization of recipes and corresponding cooking videos based on a caption-recipe alignment algorithm; the information extraction of food recipes based on lexico-syntactic patterns; and the semantic interconnection between recognized entities and web resources. The experimental results show that the proposed system is superior to existing alignment algorithms and effective in semantic cooking video annotation.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Ballan, L., Bertini, M., Bimbo, A.D., Seidenari, L., Serra, G.: Event detection and recognition for semantic annotation of video. J. Multimedia Tools and Applications 51(1), 279–302 (2011)
Bellman, S., Schweda, A., Varan, D.: A Comparison of Three Interactive Television AD Formats. Journal of Interactive Advertising 10(1), 14–34 (2009)
Cour, T., Jordan, C., Miltsakaki, E., Taskar, B.: Movie/script: alignment and parsing of video and text transcription. In: Proceeding of the 10th European Conference on Computer Vision: Part IV, pp. 158–171 (2008)
Guo, W., Diab, M.: A simple unsupervised latent semantics based approach for sentence similarity. In: Proceeding of First Joint Conference on Lexical and Computational Semantics, pp. 586–590 (2012)
Hamada, R., Okabe, J., Ide, I., Satoh, S., Sakai, S., Tanaka, H.: Cooking navi: assistant for daily cooking in kitchen. In: Proceeding 13th annual ACM International Conference on Multimedia, pp. 371–374 (2005)
Hamada, R., Miura, K., Ide, I., Satoh, S., Sakai, S., Tanaka, H.: Multimedia Integration for Cooking Video Indexing. In: Aizawa, K., Nakamura, Y., Satoh, S. (eds.) PCM 2004. LNCS, vol. 3332, pp. 657–664. Springer, Heidelberg (2004)
Homer, B.D., Plass, J.L.: Level of Interactivity and executive functions as predictors of learning in computer-based chemistry simulations. Journal of Computers in Human Behavior 26, 365–375 (2014)
Jacobs, P.S., Krupka, G.R., Rau, L.F.: Lexico-semantic pattern matching as a companion to parsing in text understanding. In: Proceeding of the Workshop on Speech and Natural Language, Collocated with the 6th Human Language Technology Conference, pp. 337–341 (1991)
Liu, Y., Liang, Y.: A Sentence Semantic Similarity Calculating Method based on Segmented Semantic Comparison. Journal of Theoretical and Applied Information Technology 48(1), 231–235 (2013)
Maynard, D., Funk, A., Peters, W.: Using lexico-syntactic ontology design patterns for ontology creation and population. In: Proceeding of the Workshop on Ontology Patterns, Collocated with the 8th International Semantic Web Conference, pp. 39–52 (2009)
Oh, K.J., Hong, M.D., Sim, S.Y., Jo, G.S.: Automatic indexing of cooking video by using caption-recipe alignment. In: Proceeding of IEEE International Conference on Behavior, Economic and Social Computing (BESC), pp. 1–6 (2014)
Panchenko, A., Morozova, O., Naets, H.: A semantic similarity measure based on lexico-syntactic patterns. In: Proceeding of the 11th Conference on Natural Language Processing (KONVENS), pp. 174–178 (2012)
Turetsky, R., Dimitrova, N.: Screenplay alignment for closed-system speaker identification and analysis of feature films. In: Proceeding of IEEE International Conference on Multimedia and Expo, pp. 1659–1662 (2004)
WireWax: Interactive Video Annotation Tools. https://www.wirewax.com
Wu, J., Worring, M.: Efficient Genre-Specific Semantic Video Indexing. IEEE Transactions on Multimedia 14(2), 291–302 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Oh, KJ., Hong, MD., Yoon, UN., Jo, GS. (2015). Text-Based Semantic Video Annotation for Interactive Cooking Videos. In: Núñez, M., Nguyen, N., Camacho, D., Trawiński, B. (eds) Computational Collective Intelligence. Lecture Notes in Computer Science(), vol 9329. Springer, Cham. https://doi.org/10.1007/978-3-319-24069-5_22
Download citation
DOI: https://doi.org/10.1007/978-3-319-24069-5_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-24068-8
Online ISBN: 978-3-319-24069-5
eBook Packages: Computer ScienceComputer Science (R0)