Chidansh A. Bhatt
Pradeep K. Atrey
Mohan S. Kankanhalli
Skip Abstract Section
Skip Supplemental Material SectionAbstract
Despite the fact that performance improvements have been reported in the last years, semantic concept detection in video remains a challenging problem. Existing concept detection techniques, with ontology rules, exploit the static correlations among primitive concepts but not the dynamic spatiotemporal correlations. The proposed method rewards (or punishes) detected primitive concepts using dynamic spatiotemporal correlations of the given ontology rules and updates these ontology rules based on the accuracy of detection. Adaptively learned ontology rules significantly help in improving the overall accuracy of concept detection as shown in the experimental result.
Supplemental Material
Available for Download
zip
bhatt.zip (30.7 KB)
Supplemental movie, appendix, image and software files for, A reward-and-punishment-based approach for concept detection using adaptive ontology rules
- Amir, A., Berg, M., Chang, S.-F., Iyengar, G., Lin, C.-Y., Natsev, A., Neti, C., Nock, H., Naphade, M., Hsu, W., Smith, J. R., Tseng, B., Wu, Y., Zhang, D., and Watson, I. T. J. 2003. Ibm research trecvid-2003 video retrieval system. In Proceedings of the TREC Video Retrieval Evaluation (NIST TRECVID'03).Google Scholar
- Assfalg, J., Bertini, M., Colombo, C., Bimbo, A. D., and Nunziati, W. 2003. Semantic annotation of soccer videos: Automatic highlights identification. Comput. Vis. Image Understand. 92, 2--3, 285--305. Google ScholarDigital Library
- Bai, L., Lao, S., Zhang, W., Jones, G. J., and Smeaton, A. F. 2007. Video semantic content analysis based on ontology combinedmpeg-7. In Adaptive Multimedial Retrieval: Retrieval, User, and Semantics, Lecture Notes in Computer Science, vol. 4918, Springer, 237--250.Google Scholar
- Ballan, L., Bertini, M., Bimbo, A., Seidenari, L., and Serra, G. 2011. Event detection and recognition for semantic annotation of video. Multimedia Tools Appl. 51, 1. Google ScholarDigital Library
- Ballan, L., Bertini, M., Bimbo, A. D., and Serra, G. 2010. Video annotation and retrieval using ontologies and rule learning. IEEE Multimedia 17, 4. Google ScholarDigital Library
- Bather, J. 2000. Decision Theory: An Introduction to Dynamic Programming and Sequential Decisions. John Wiley & Sons. Google ScholarDigital Library
- Bertini, M., Cucchiara, R., del Bimbo, A., and Torniai, C. 2005. Video annotation with pictorially enriched ontologies. In Proceedings of the IEEE International Conference on Multimedia and Expo.Google Scholar
- Bhatt, C. and Kankanhalli, M. 2011. Multimedia data mining: State of the art and challenges. Multimedia Tools Appl. 51, 1. Google ScholarDigital Library
- Brand, M. and Kettnaker, V. 2000. Discovery and segmentation of activities in video. IEEE Trans. Pattern Anal. Mach. Intell. 22, 8, 844--851. Google ScholarDigital Library
- Castano, S., Espinosa, S., Ferrara, A., Karkaletsis, V., Kaya, A., Melzer, S., Moller, R., Montanelli, S., and Petasis, G. 2007. Ontology dynamics with multimedia information: The boemie evolution methodology. In Proceedings of the International Workshop on Ontology Dynamics.Google Scholar
- Castano, S., Espinosa, S., Ferrara, A., Karkaletsis, V., Kaya, A., Moller, R., Montanelli, S., Petasis, G., and Wessel, M. 2008. Multimedia interpretation for dynamic ontology evolution. J. Logic Comput. 19, 5, 859--897. Google ScholarDigital Library
- Chao, C.-Y., Shih, H.-C., and Huang, C.-L. 2005. Semantics-Based highlight extraction of soccer program using dbn. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing.Google Scholar
- Dasiopoulou, S., Kompatsiaris, I., and Strintzis, M. 2010. Investigating fuzzy dls-based reasoning in semantic image analysis. Multimedia Tools Appl. 49, 1. Google ScholarDigital Library
- Dasiopoulou, S., Mezaris, V., Kompatsiaris, I., Papastathis, V. K., and Strintzis, M. G. 2005. Knowledge-Assisted semantic video object detection. Trans. Circ. Syst. Video Technol. 15, 10. Google ScholarDigital Library
- Everingham, M., van Gool, L., Williams, C. K. I., Winn, J., and Zisserman, A. 2011. The PASCAL visual object classes challenge 2011 (VOC2011) results. http://www.pascal-network.org/challenges/VOC/voc2011/workshop/index.html.Google Scholar
- Harte, N., Lennon, D., and Kokaram, A. 2009. On parsing visual sequences with the hidden Markov model. J. Image Video Process. 6:1--6:13. Google ScholarDigital Library
- Haubold, A. and Naphade, M. 2007. Classification of video events using 4-dimensional time-compressed motion features. In Proceedings of the ACM International Conference on Image and Video Retrieval. 178--185. Google ScholarDigital Library
- Hossain, M. A., Atrey, P. K., and Saddik, A. E. 2009. Learning multisensor confidence using a reward-and-punishment mechanism. IEEE Trans. Instrument. Measur. 58, 5.Google ScholarCross Ref
- Kohlmorgen, J., Lemm, S., Muller, K., Liehr, S., and Pawelzik, K. 1999. Fast change point detection in switching dynamics using a hidden Markov model of prediction experts. In Proceedings of the 9th International Conference on Artificial Neural Networks. Vol. 1. 204--209.Google Scholar
- Li, L., Prakash, B. A., and Faloutsos, C. 2010. Parsimonious linear fingerprinting for time series. Proc. VLDB Endow. 3, 1. Google ScholarDigital Library
- Oca, V. M. D., Jeske, D. R., Zhang, Q., Rendon, C., and Marvasti, M. 2010. A cusum change-point detection algorithm for non-stationary sequences with application to data network surveillance. J. Syst. Softw. 83, 7. Google ScholarDigital Library
- Over, P., Awad, G., Michel, M., Fiscus, J., Kraaij, W., Smeaton, A. F., and Quenot, G. 2011. Trecvid 2011 -- An overview of the goals, tasks, data, evaluation mechanisms and metrics. In Proceedings of the TREC Video Retrieval Evaluation (TRECVID'11) Workshop.Google Scholar
- Petridis, S. and Perantonis, S. J. 2011. Semantics extraction from multimedia data: An ontology-based machine learning approach. In Perception-Action Cycle, Series in Cognitive and Neural Systems, Springer.Google Scholar
- Qi, G.-J., Hua, X.-S., Rui, Y., Tang, J., Mei, T., and Zhang, H.-J. 2007. Correlative multi-label video annotation. In Proceedings of the ACM International Conference on Multimedia. 17--26. Google ScholarDigital Library
- Sadlier, D. and O'Connor, N. 2005. Event detection in field sports video using audio-visual features and a support vector machine. IEEE Trans. Circ. Syst. Video Technol. 15, 10, 1225--1233. Google ScholarDigital Library
- Shyu, M.-L., Xie, Z., Chen, M., and Chen, S.-C. 2008. Video semantic event/concept detection using a subspace-based multimedia data mining framework. IEEE Trans. Multimedia 10, 2. Google ScholarDigital Library
- Smeaton, A. F., Over, P., and Kraaij, W. 2006. Evaluation campaigns and trecvid. In Proceedings of the ACM International Workshop on Multimedia Information Retrieval. 321--330. Google ScholarDigital Library
- Smith, J. R., Naphade, M., and Natsev, A. 2003. Multimedia semantic indexing using model vectors. In Proceedings of the IEEE International Conference on Multimedia and Expo. 445--448. Google ScholarDigital Library
- Wang, M., Hua, X.-S., Hong, R., Tang, J., Qi, G.-J., and Song, Y. 2009a. Unified video annotation via multigraph learning. IEEE Trans. Cir. Syst. Video Technol. 19, 5, 733--746. Google ScholarDigital Library
- Wang, M., Hua, X.-S., Tang, J., and Hong, R. 2009b. Beyond distance measurement: constructing neighborhood similarity for video annotation. IEEE Trans. Multimedia 11, 3, 465--476. Google ScholarDigital Library
- Wu, Y., Tseng, B. L., and Smith, J. R. 2004. Ontology-Based multi-classification learning for video concept detection. In Proceedings of the International Conference on Multimedia and Expo.Google Scholar
- Xu, D. and Chang, S.-F. 2008. Video event recognition using Kernel methods with multilevel temporal alignment. IEEE Trans. Pattern Anal. Mach. Intell. 30, 11, 1985--1997. Google ScholarDigital Library
- Xu, P., Xie, L., Chang, S.-F., Divakaran, A., Vetro, A., and Sun, H. 2001. Algorithms and system for segmentation and structure analysis in soccer video. In Proceedings of the International Conference on Multimedia and Expo.Google Scholar
- Yan, R., Tesic, J., and Smith, J. R. 2007. Model-Shared subspace boosting for multi-label classification. In Proceedings of the ACM International Conference on Knowledge Discovery and Data Mining. 834--843. Google ScholarDigital Library
- Yanagawa, A., Chang, S.-F., Kennedy, L., and Hsu, W. 2007. Columbia university's baseline detectors for 374 lscom semantic visual concepts. Tech. rep. 222-2006, Columbia University.Google Scholar
- Yang, K. and Shahabi, C. 2004. A pca-based similarity measure for multivariate time series. In Proceedings of the ACM International Workshop on Multimedia Databases. Google ScholarDigital Library
- Zha, Z.-J., Mei, T., Wang, Z., and Hua, X.-S. 2007. Building a comprehensive ontology to refine video concept detection. In Proceedings of the ACM International Workshop on Multimedia Information Retrieval. Google ScholarDigital Library
- Zhou, X., Zhuang, X., Yan, S., Chang, S.-F., Hasegawa-Johnson, M., and Huang, T. S. 2008. Sift-Bag kernel for video event analysis. In Proceedings of the ACM International Conference on Multimedia. Google ScholarDigital Library
Index Terms
- A reward-and-punishment-based approach for concept detection using adaptive ontology rules
Recommendations
Video semantic concept detection using ontology
ICIMCS '11: Proceedings of the Third International Conference on Internet Multimedia Computing and ServiceSemantic concept detection in video is a challenge for video semantic content analysis. The performance of semantic concept detection methods depends on representing the video semantic content exactly. In this paper, perception concept and semantic ...
Ontology-Based Inter-concept Relation Fusion for Concept Detection
PCM '08: Proceedings of the 9th Pacific Rim Conference on Multimedia: Advances in Multimedia Information ProcessingAlthough detectors for individual concepts have been widely studied in multimedia search area, the exploration of inter-concept relations among concepts receives relatively less attention, especially when hierarchical concept taxonomy is not manually ...
Markov Random Field for Image Concept Detection
ISCID '14: Proceedings of the 2014 Seventh International Symposium on Computational Intelligence and Design - Volume 02Recent years have witnessed phenomenal growth in the number of internet multimedia documents, such as un-annotated scene images. Image concept detection is an important step for image query. In this paper, we tackle the problem of image concept ...
Comments