Abstract
Unsupervised video anomaly detection approaches often demand complex models and substantial computational resources for effective performance. In contrast, we introduce a supervised and end-to-end trainable deep learning approach that leverages both performance and computational efficiency by harnessing frame-level annotated data. The framework begins with the utilization of an Inception encoder network in the initial stage to learn feature representations. Notably, the Inception network’s proficiency in capturing intricate and high-level features in frames seamlessly extends to the analysis of video data. By using these extracted features, the model excels in identifying deviations from learned patterns, making it highly adept at detecting anomalies in video sequences. The subsequent stage involves a sequence of fully connected layers followed by a classifier that is responsible for classifying input frames as either normal or anomalous based on the extracted features. To thoroughly validate this methodology, extensive experiments are carried out on widely used benchmark datasets. These evaluations involved comprehensive comparisons with contemporary approaches in the field. The experimental findings consistently validate the efficacy and efficiency of the proposed approach, underscoring its outstanding accuracy in identifying anomalies. Additionally, the approach operates with significantly reduced computational overhead, rendering it an appealing solution for real-world applications that demand timely and precise anomaly detection.











Similar content being viewed by others
Availability of data and materials
The data used in this study are available upon request.
Code availability
The code used for data analysis is available upon request.
References
Sikdar A, Chowdhury AS (2020) An adaptive training-less framework for anomaly detection in crowd scenes. Neurocomputing 415:317–331. https://doi.org/10.1016/j.neucom.2020.07.058
Hasan M, Choi J, Neumann J, Roy-Chowdhury AK, Davis LS (2016) Learning temporal regularity in video sequences. In: 2016 IEEE Conference on computer vision and pattern recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pp 733–742. https://doi.org/10.1109/CVPR.2016.86
Chong YS, Tay YH (2017) Abnormal event detection in videos using spatiotemporal autoencoder. In: Cong F, Leung AC, Wei Q (eds) Advances in neural networks - ISNN 2017 - 14th International symposium, ISNN 2017, Sapporo, Hakodate, and Muroran, Hokkaido, Japan, June 21-26, 2017, Proceedings, Part II. Lecture Notes in Computer Science, vol 10262, pp 189–196. https://doi.org/10.1007/978-3-319-59081-3_23
Li N, Chang F, Liu C (2021) Spatial-temporal cascade autoencoder for video anomaly detection in crowded scenes. IEEE Trans Multim 23:203–215. https://doi.org/10.1109/TMM.2020.2984093
Kommanduri R, Ghorai M (2023) Bi-read: Bi-residual autoencoder based feature enhancement for video anomaly detection. J Vis Commun Image Represent 95:103860. https://doi.org/10.1016/j.jvcir.2023.103860
Akcay S, Abarghouei AA, Breckon TP (2018) Ganomaly: Semi-supervised anomaly detection via adversarial training. In: Jawahar CV, Li H, Mori G, Schindler K (eds) Computer vision - ACCV 2018 - 14th Asian conference on computer vision, Perth, Australia, December 2-6, 2018, Revised Selected Papers, Part III. Lecture Notes in Computer Science, vol 11363, pp 622–637. https://doi.org/10.1007/978-3-030-20893-6_39
Chen D, Yue L, Chang X, Xu M, Jia T (2021) NM-GAN: noise-modulated generative adversarial network for video anomaly detection. Pattern Recognit 116:107969. https://doi.org/10.1016/j.patcog.2021.107969
Luo W, Liu W, Lian D, Gao S (2022) Future frame prediction network for video anomaly detection. IEEE Trans Pattern Anal Mach Intell 44(11):7505–7520. https://doi.org/10.1109/TPAMI.2021.3129349
Ionescu RT, Khan FS, Georgescu M, Shao L (2019) Object-centric auto-encoders and dummy anomalies for abnormal event detection in video. In: IEEE Conference on computer vision and pattern recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pp 7842–7851. https://doi.org/10.1109/CVPR.2019.00803
Bansod SD, Nandedkar AV (2019) Transfer learning for video anomaly detection. J Intell Fuzzy Syst 36(3):1967–1975. https://doi.org/10.3233/JIFS-169908
Lalit R, Purwar RK, Verma S, Jain A (2022) Correction to: Crowd abnormality detection in video sequences using supervised convolutional neural network. Multim Tools Appl 81(22):32701. https://doi.org/10.1007/s11042-022-13375-0
Ramachandra B, Jones MJ, Vatsavai RR (2022) A survey of single-scene video anomaly detection. IEEE Trans Pattern Anal Mach Intell 44(5):2293–2312. https://doi.org/10.1109/TPAMI.2020.3040591
Mondal R, Chanda B (2018) Anomaly detection using context dependent optical flow. In: ICVGIP 2018: 11th Indian conference on computer vision, graphics and image processing, Hyderabad, India, 18-22 December, 2018, pp 22–1228. https://doi.org/10.1145/3293353.3293375
Gandapur MQ (2022) E2E-VSDL: end-to-end video surveillance-based deep learning model to detect and prevent criminal activities. Image Vis Comput 123:104467. https://doi.org/10.1016/j.imavis.2022.104467
Amin J, Anjum MA, Ibrar K, Sharif M, Kadry S, Crespo RG (2023) Detection of anomaly in surveillance videos using quantum convolutional neural networks. Image Vis Comput 135:104710. https://doi.org/10.1016/j.imavis.2023.104710
Park C, Cho M, Lee M, Lee S (2022) Fastano: Fast anomaly detection via spatio-temporal patch transformation. In: IEEE/CVF Winter conference on applications of computer vision, WACV 2022, Waikoloa, HI, USA, January 3-8, 2022, pp 1908–1918. https://doi.org/10.1109/WACV51458.2022.00197
Liu Y, Liu J, Zhao M, Li S, Song L (2022) Collaborative normality learning framework for weakly supervised video anomaly detection. IEEE Trans Circuits Syst II Express Briefs 69(5):2508–2512. https://doi.org/10.1109/TCSII.2022.3161061
Sultani W, Chen C, Shah M (2018) Real-world anomaly detection in surveillance videos. In: 2018 IEEE Conference on computer vision and pattern recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pp 6479–6488. https://doi.org/10.1109/CVPR.2018.00678
Zhu Y, Newsam SD (2019) Motion-aware feature for improved video anomaly detection. In: 30th British machine vision conference 2019, BMVC 2019, Cardiff, UK, September 9-12, 2019, p 270
Zhong J, Li N, Kong W, Liu S, Li TH, Li G (2019) Graph convolutional label noise cleaner: Train a plug-and-play action classifier for anomaly detection. In: IEEE Conference on computer vision and pattern recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pp 1237–1246. https://doi.org/10.1109/CVPR.2019.00133
Thakare KV, Raghuwanshi Y, Dogra DP, Choi H, Kim I (2023) Dyannet: A scene dynamicity guided self-trained video anomaly detection network. In: IEEE/CVF Winter conference on applications of computer vision, WACV 2023, Waikoloa, HI, USA, January 2-7, 2023, pp 5530–5539. https://doi.org/10.1109/WACV56688.2023.00550
Zhou S, Shen W, Zeng D, Fang M, Wei Y, Zhang Z (2016) Spatial-temporal convolutional neural networks for anomaly detection and localization in crowded scenes. Signal Process: Image Commun 47:358–368. https://doi.org/10.1016/j.image.2016.06.007
Hinami R, Mei T, Satoh S (2017) Joint detection and recounting of abnormal events by learning deep generic knowledge. In: IEEE International conference on computer vision, ICCV 2017, Venice, Italy, October 22-29, 2017, pp 3639–3647. https://doi.org/10.1109/ICCV.2017.391
Ruchika P (2019) Abnormality detection using lbp features and k-means labelling based feed-forward neural network in video sequence. Int J Innovative Technol Exploring Eng 8:629–633
Ma Q (2021) Abnormal event detection in videos based on deep neural networks. Sci Program 2021:6412608–164126088. https://doi.org/10.1155/2021/6412608
Kommanduri R, Ghorai M (2023) A supervised approach for efficient video anomaly detection using transfer learning, pp 209–217. https://doi.org/10.1007/978-3-031-45170-6_22
Liu D, Cui Y, Yan L, Mousas C, Yang B, Chen Y (2021) Densernet: Weakly supervised visual localization using multi-scale feature aggregation. Proceedings of the AAAI Conference on Artificial Intelligence 35(7):6101–6109. https://doi.org/10.1609/aaai.v35i7.16760
Liu D, Cui Y, Tan W, Chen Y (2021) Sg-net: Spatial granularity network for one-stage video instance segmentation. In: 2021 IEEE/CVF conference on computer vision and pattern recognition (CVPR). https://doi.org/10.1109/cvpr46437.2021.00969
Liu D, Liang J, Geng T, Loui A, Zhou T (2023) Tripartite feature enhanced pyramid network for dense prediction. IEEE Trans Image Process 32:2678–2692. https://doi.org/10.1109/tip.2023.3272826
Xu D, Ricci E, Yan Y, Song J, Sebe N (2015) Learning deep representations of appearance and motion for anomalous event detection. In: Procedings of the British machine vision conference 2015. BMVC 2015. https://doi.org/10.5244/c.29.8
He C, Shao J, Sun J (2017) An anomaly-introduced learning method for abnormal event detection. Multimed Tools Appl 77(22):29573–29588. https://doi.org/10.1007/s11042-017-5255-z
Zhou JT, Du J, Zhu H, Peng X, Liu Y, Goh RSM (2019) Anomalynet: An anomaly detection network for video surveillance. IEEE Trans Inf Forensics Secur 14(10):2537–2550. https://doi.org/10.1109/TIFS.2019.2900907
Fang Z, Liang J, Zhou JT, Xiao Y, Yang F (2022) Anomaly detection with bidirectional consistency in videos. IEEE Trans Neural Netw Learn Syst 33(3):1079–1092. https://doi.org/10.1109/tnnls.2020.3039899
Tian Y, Pang G, Chen Y, Singh R, Verjans JW, Carneiro G (2021) Weakly-supervised video anomaly detection with robust temporal feature magnitude learning. In: 2021 IEEE/CVF International conference on computer vision (ICCV). https://doi.org/10.1109/iccv48922.2021.00493
Sabih M, Vishwakarma DK (2022) A novel framework for detection of motion and appearance-based anomaly using ensemble learning and lstms. Expert Syst Appl 192:116394. https://doi.org/10.1016/j.eswa.2021.116394
Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-cam: visual explanations from deep networks via gradient-based localization. In: 2017 IEEE International conference on computer vision (ICCV). https://doi.org/10.1109/iccv.2017.74
Brown TB, Mané D, Roy A, Abadi M, Gilmer J (2017) Adversarial patch. arXiv preprint arXiv:1712.09665. https://doi.org/10.48550/arXiv.1712.09665
Cheng Z, Liang J, Choi H, Tao G, Cao Z, Liu D, Zhang X (2022) Physical attack on monocular depth estimation with optimal adversarial patches, pp 514–532 . https://doi.org/10.1007/978-3-031-19839-7_30
Funding
No external funding was received for this research.
Author information
Authors and Affiliations
Contributions
All authors contributed equally to the research and manuscript preparation. Each author played a significant role in the design of the study, data collection, data analysis, and manuscript writing.
Corresponding author
Ethics declarations
Consent to participate
All participants provided informed consent to participate in the study.
Consent for publication
All authors consent to the publication of this manuscript.
Ethics approval
This research study was conducted in compliance with all relevant ethical standards and guidelines.
Conflict of Interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Kommanduri, R., Ghorai, M. Anomaly detection in video surveillance: a supervised inception encoder approach. Multimed Tools Appl 83, 78517–78534 (2024). https://doi.org/10.1007/s11042-024-18604-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-024-18604-2