Abstract
Video scene retrieval and anomaly detection are important problems in the area of computer vision that share a common concept, of vectorization of the input image frames. We propose here a new vectorization approach and a fast object tracking algorithm. First step is to use any of the existing methods for recognition of objects in image frames. The subsequent step is our key contribution, to use information inside these objects, tracking and generation of semantic vectors for a sequence of image frames. We introduce a novel way of ultra high speed object tracking using a density based clustering of local vector representation of objects across video frames. The vectorization results in semantic features involve object types, their identifiers and movement information. The algorithm has been validated for its ability to retrieve scenes having the highest similarity, over a subset of the YouTube 8M data set on about 1200 videos having 36651 sub-scenes. One type of validation is, among the frames that were closest to the query scene, the fraction of them having common picture characteristics is 90%. The second type of validation is, among the successive frames with a little time gap, a similarity of more than 90% is recorded as desired. The vectorization algorithm is tested for its usefulness to a different problem scenario of anomaly detection in video frames. The vectorization performed well for qualitative and quantitative evaluation on standard and customized anomaly detection data sets of videos and results are reported here.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abu-El-Haija, S., et al.: Youtube-8M: a large-scale video classification benchmark. CoRR abs/1609.08675
Altadmri, A., Ahmed, A.: A framework for automatic semantic video annotation. Multimed. Tools Appl. 72, 1167–1191 (2014). https://doi.org/10.1007/s11042-013-1363-6
Basharat, A., Zhai, Y., Shah, M.: Content based video matching using spatiotemporal volumes. Comput. Vis. Image Underst. 110(3), 360–377 (2008). Similarity Matching in Computer Vision and Multimedia
Bentley, J.L.: Multidimensional binary search trees used for associative searching. Commun. ACM 18(9), 509–517 (1975)
Buchanan, A., Fitzgibbon, A.: Interactive feature tracking using K-D trees and dynamic programming. In: VPR 2006, vol. 1, pp. 626–633 (2006)
Hasan, M., Choi, J., Neumann, J., Roy-Chowdhury, A., Davis, L.: Learning temporal regularity in video sequences, pp. 733–742, June 2016
Zhang, H.Z., Wang, J.Y.A., Altunbasak, Y.: Content-based video retrieval and compression: a unified solution. In: Proceedings of International Conference on Image Processing, vol. 1, pp. 13–16 (1997)
Jawahar, C.V., Chennupati, J.B., Paluri, B., Jammalamadaka, N.: Video retrieval based on textual queries (2005)
Jiang, R.M., Crookes, D.: Approach to automatic video motion segmentation. Electron. Lett. 43(18), 968–970 (2007)
Kim, J., Grauman, K.: Observe locally, infer globally: a space-time MRF for detecting abnormal activities with incremental updates. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2928 (2009)
Kubica, J., Masiero, J., Moore, A., Jedicke, R., Connolly, A.: Variable KD-tree algorithms for spatial pattern search and discovery (2005)
Liu, Y., Zhang, D., Lu, G., Ma, W.Y.: A survey of content-based image retrieval with high-level semantics. Pattern Recogn. 40(1), 262–282 (2007)
Lu, C., Shi, J., Jia, J.: Abnormal event detection at 150 fps in MATLAB. In: 2013 IEEE International Conference on Computer Vision, pp. 2720–2727 (2013)
Mahadevan, V., Li, W., Bhalodia, V., Vasconcelos, N.: Anomaly detection in crowded scenes. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1975–1981 (2010)
Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using social force model. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 935–942 (2009)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779–788 (2016)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems 28, pp. 91–99. Curran Associates Inc
Reynolds, D.: Gaussian mixture models. In: Li, S.Z., Jain, A. (eds.) Encyclopedia of Biometrics, pp. 827–832. Springer, Boston (2009). https://doi.org/10.1007/978-0-387-73003-5_196
Zaheer, M.Z., Lee, J.H., Lee, S., Seo, B.: A brief survey on contemporary methods for anomaly detection in videos. In: 2019 International Conference on Information and Communication Technology Convergence (ICTC), pp. 472–473 (2019)
Zhang, X.P., Chen, Z.: An automated video object extraction system based on spatiotemporal independent component analysis and multiscale segmentation. EURASIP J. Adv. Sig. Process. 2006, 1–22 (2006). https://doi.org/10.1155/ASP/2006/45217
Zhou, H., Yuan, Y., Shi, C.: Object tracking using sift features and mean shift. Comput. Vis. Image Underst. 113(3), 345–352 (2009). https://doi.org/10.1016/j.cviu.2008.08.006. http://www.sciencedirect.com/science/article/pii/S1077314208001331. Special Issue on Video Analysis
Acknowledgements
Acknowledging herewith research grant and support from Toshiba software (India) Pvt. Ltd., as part of project RB/1920/CSE/001/TOSH/KALI hosted at IIT Tirupati.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Prashanth, K., Kalidas, Y., Kumar, J.R.B., Ayyagari, S.P.K., Deep, A. (2021). An Algorithm for Semantic Vectorization of Video Scenes - Applications to Retrieval and Anomaly Detection. In: Singh, S.K., Roy, P., Raman, B., Nagabhushan, P. (eds) Computer Vision and Image Processing. CVIP 2020. Communications in Computer and Information Science, vol 1378. Springer, Singapore. https://doi.org/10.1007/978-981-16-1103-2_31
Download citation
DOI: https://doi.org/10.1007/978-981-16-1103-2_31
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-1102-5
Online ISBN: 978-981-16-1103-2
eBook Packages: Computer ScienceComputer Science (R0)