Skip to main content
Log in

Anomaly detection in video surveillance: a supervised inception encoder approach

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Unsupervised video anomaly detection approaches often demand complex models and substantial computational resources for effective performance. In contrast, we introduce a supervised and end-to-end trainable deep learning approach that leverages both performance and computational efficiency by harnessing frame-level annotated data. The framework begins with the utilization of an Inception encoder network in the initial stage to learn feature representations. Notably, the Inception network’s proficiency in capturing intricate and high-level features in frames seamlessly extends to the analysis of video data. By using these extracted features, the model excels in identifying deviations from learned patterns, making it highly adept at detecting anomalies in video sequences. The subsequent stage involves a sequence of fully connected layers followed by a classifier that is responsible for classifying input frames as either normal or anomalous based on the extracted features. To thoroughly validate this methodology, extensive experiments are carried out on widely used benchmark datasets. These evaluations involved comprehensive comparisons with contemporary approaches in the field. The experimental findings consistently validate the efficacy and efficiency of the proposed approach, underscoring its outstanding accuracy in identifying anomalies. Additionally, the approach operates with significantly reduced computational overhead, rendering it an appealing solution for real-world applications that demand timely and precise anomaly detection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Availability of data and materials

The data used in this study are available upon request.

Code availability

The code used for data analysis is available upon request.

Notes

  1. https://github.com/UjjwalSaxena/Automold--Road-Augmentation-Library

References

  1. Sikdar A, Chowdhury AS (2020) An adaptive training-less framework for anomaly detection in crowd scenes. Neurocomputing 415:317–331. https://doi.org/10.1016/j.neucom.2020.07.058

    Article  Google Scholar 

  2. Hasan M, Choi J, Neumann J, Roy-Chowdhury AK, Davis LS (2016) Learning temporal regularity in video sequences. In: 2016 IEEE Conference on computer vision and pattern recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pp 733–742. https://doi.org/10.1109/CVPR.2016.86

  3. Chong YS, Tay YH (2017) Abnormal event detection in videos using spatiotemporal autoencoder. In: Cong F, Leung AC, Wei Q (eds) Advances in neural networks - ISNN 2017 - 14th International symposium, ISNN 2017, Sapporo, Hakodate, and Muroran, Hokkaido, Japan, June 21-26, 2017, Proceedings, Part II. Lecture Notes in Computer Science, vol 10262, pp 189–196. https://doi.org/10.1007/978-3-319-59081-3_23

  4. Li N, Chang F, Liu C (2021) Spatial-temporal cascade autoencoder for video anomaly detection in crowded scenes. IEEE Trans Multim 23:203–215. https://doi.org/10.1109/TMM.2020.2984093

    Article  Google Scholar 

  5. Kommanduri R, Ghorai M (2023) Bi-read: Bi-residual autoencoder based feature enhancement for video anomaly detection. J Vis Commun Image Represent 95:103860. https://doi.org/10.1016/j.jvcir.2023.103860

    Article  Google Scholar 

  6. Akcay S, Abarghouei AA, Breckon TP (2018) Ganomaly: Semi-supervised anomaly detection via adversarial training. In: Jawahar CV, Li H, Mori G, Schindler K (eds) Computer vision - ACCV 2018 - 14th Asian conference on computer vision, Perth, Australia, December 2-6, 2018, Revised Selected Papers, Part III. Lecture Notes in Computer Science, vol 11363, pp 622–637. https://doi.org/10.1007/978-3-030-20893-6_39

  7. Chen D, Yue L, Chang X, Xu M, Jia T (2021) NM-GAN: noise-modulated generative adversarial network for video anomaly detection. Pattern Recognit 116:107969. https://doi.org/10.1016/j.patcog.2021.107969

    Article  Google Scholar 

  8. Luo W, Liu W, Lian D, Gao S (2022) Future frame prediction network for video anomaly detection. IEEE Trans Pattern Anal Mach Intell 44(11):7505–7520. https://doi.org/10.1109/TPAMI.2021.3129349

    Article  Google Scholar 

  9. Ionescu RT, Khan FS, Georgescu M, Shao L (2019) Object-centric auto-encoders and dummy anomalies for abnormal event detection in video. In: IEEE Conference on computer vision and pattern recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pp 7842–7851. https://doi.org/10.1109/CVPR.2019.00803

  10. Bansod SD, Nandedkar AV (2019) Transfer learning for video anomaly detection. J Intell Fuzzy Syst 36(3):1967–1975. https://doi.org/10.3233/JIFS-169908

    Article  Google Scholar 

  11. Lalit R, Purwar RK, Verma S, Jain A (2022) Correction to: Crowd abnormality detection in video sequences using supervised convolutional neural network. Multim Tools Appl 81(22):32701. https://doi.org/10.1007/s11042-022-13375-0

    Article  Google Scholar 

  12. Ramachandra B, Jones MJ, Vatsavai RR (2022) A survey of single-scene video anomaly detection. IEEE Trans Pattern Anal Mach Intell 44(5):2293–2312. https://doi.org/10.1109/TPAMI.2020.3040591

    Article  Google Scholar 

  13. Mondal R, Chanda B (2018) Anomaly detection using context dependent optical flow. In: ICVGIP 2018: 11th Indian conference on computer vision, graphics and image processing, Hyderabad, India, 18-22 December, 2018, pp 22–1228. https://doi.org/10.1145/3293353.3293375

  14. Gandapur MQ (2022) E2E-VSDL: end-to-end video surveillance-based deep learning model to detect and prevent criminal activities. Image Vis Comput 123:104467. https://doi.org/10.1016/j.imavis.2022.104467

    Article  Google Scholar 

  15. Amin J, Anjum MA, Ibrar K, Sharif M, Kadry S, Crespo RG (2023) Detection of anomaly in surveillance videos using quantum convolutional neural networks. Image Vis Comput 135:104710. https://doi.org/10.1016/j.imavis.2023.104710

    Article  Google Scholar 

  16. Park C, Cho M, Lee M, Lee S (2022) Fastano: Fast anomaly detection via spatio-temporal patch transformation. In: IEEE/CVF Winter conference on applications of computer vision, WACV 2022, Waikoloa, HI, USA, January 3-8, 2022, pp 1908–1918. https://doi.org/10.1109/WACV51458.2022.00197

  17. Liu Y, Liu J, Zhao M, Li S, Song L (2022) Collaborative normality learning framework for weakly supervised video anomaly detection. IEEE Trans Circuits Syst II Express Briefs 69(5):2508–2512. https://doi.org/10.1109/TCSII.2022.3161061

    Article  Google Scholar 

  18. Sultani W, Chen C, Shah M (2018) Real-world anomaly detection in surveillance videos. In: 2018 IEEE Conference on computer vision and pattern recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pp 6479–6488. https://doi.org/10.1109/CVPR.2018.00678

  19. Zhu Y, Newsam SD (2019) Motion-aware feature for improved video anomaly detection. In: 30th British machine vision conference 2019, BMVC 2019, Cardiff, UK, September 9-12, 2019, p 270

  20. Zhong J, Li N, Kong W, Liu S, Li TH, Li G (2019) Graph convolutional label noise cleaner: Train a plug-and-play action classifier for anomaly detection. In: IEEE Conference on computer vision and pattern recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pp 1237–1246. https://doi.org/10.1109/CVPR.2019.00133

  21. Thakare KV, Raghuwanshi Y, Dogra DP, Choi H, Kim I (2023) Dyannet: A scene dynamicity guided self-trained video anomaly detection network. In: IEEE/CVF Winter conference on applications of computer vision, WACV 2023, Waikoloa, HI, USA, January 2-7, 2023, pp 5530–5539. https://doi.org/10.1109/WACV56688.2023.00550

  22. Zhou S, Shen W, Zeng D, Fang M, Wei Y, Zhang Z (2016) Spatial-temporal convolutional neural networks for anomaly detection and localization in crowded scenes. Signal Process: Image Commun 47:358–368. https://doi.org/10.1016/j.image.2016.06.007

    Article  Google Scholar 

  23. Hinami R, Mei T, Satoh S (2017) Joint detection and recounting of abnormal events by learning deep generic knowledge. In: IEEE International conference on computer vision, ICCV 2017, Venice, Italy, October 22-29, 2017, pp 3639–3647. https://doi.org/10.1109/ICCV.2017.391

  24. Ruchika P (2019) Abnormality detection using lbp features and k-means labelling based feed-forward neural network in video sequence. Int J Innovative Technol Exploring Eng 8:629–633

    Article  Google Scholar 

  25. Ma Q (2021) Abnormal event detection in videos based on deep neural networks. Sci Program 2021:6412608–164126088. https://doi.org/10.1155/2021/6412608

    Article  Google Scholar 

  26. Kommanduri R, Ghorai M (2023) A supervised approach for efficient video anomaly detection using transfer learning, pp 209–217. https://doi.org/10.1007/978-3-031-45170-6_22

  27. Liu D, Cui Y, Yan L, Mousas C, Yang B, Chen Y (2021) Densernet: Weakly supervised visual localization using multi-scale feature aggregation. Proceedings of the AAAI Conference on Artificial Intelligence 35(7):6101–6109. https://doi.org/10.1609/aaai.v35i7.16760

    Article  Google Scholar 

  28. Liu D, Cui Y, Tan W, Chen Y (2021) Sg-net: Spatial granularity network for one-stage video instance segmentation. In: 2021 IEEE/CVF conference on computer vision and pattern recognition (CVPR). https://doi.org/10.1109/cvpr46437.2021.00969

  29. Liu D, Liang J, Geng T, Loui A, Zhou T (2023) Tripartite feature enhanced pyramid network for dense prediction. IEEE Trans Image Process 32:2678–2692. https://doi.org/10.1109/tip.2023.3272826

    Article  Google Scholar 

  30. Xu D, Ricci E, Yan Y, Song J, Sebe N (2015) Learning deep representations of appearance and motion for anomalous event detection. In: Procedings of the British machine vision conference 2015. BMVC 2015. https://doi.org/10.5244/c.29.8

  31. He C, Shao J, Sun J (2017) An anomaly-introduced learning method for abnormal event detection. Multimed Tools Appl 77(22):29573–29588. https://doi.org/10.1007/s11042-017-5255-z

    Article  Google Scholar 

  32. Zhou JT, Du J, Zhu H, Peng X, Liu Y, Goh RSM (2019) Anomalynet: An anomaly detection network for video surveillance. IEEE Trans Inf Forensics Secur 14(10):2537–2550. https://doi.org/10.1109/TIFS.2019.2900907

    Article  Google Scholar 

  33. Fang Z, Liang J, Zhou JT, Xiao Y, Yang F (2022) Anomaly detection with bidirectional consistency in videos. IEEE Trans Neural Netw Learn Syst 33(3):1079–1092. https://doi.org/10.1109/tnnls.2020.3039899

    Article  Google Scholar 

  34. Tian Y, Pang G, Chen Y, Singh R, Verjans JW, Carneiro G (2021) Weakly-supervised video anomaly detection with robust temporal feature magnitude learning. In: 2021 IEEE/CVF International conference on computer vision (ICCV). https://doi.org/10.1109/iccv48922.2021.00493

  35. Sabih M, Vishwakarma DK (2022) A novel framework for detection of motion and appearance-based anomaly using ensemble learning and lstms. Expert Syst Appl 192:116394. https://doi.org/10.1016/j.eswa.2021.116394

    Article  Google Scholar 

  36. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-cam: visual explanations from deep networks via gradient-based localization. In: 2017 IEEE International conference on computer vision (ICCV). https://doi.org/10.1109/iccv.2017.74

  37. Brown TB, Mané D, Roy A, Abadi M, Gilmer J (2017) Adversarial patch. arXiv preprint arXiv:1712.09665. https://doi.org/10.48550/arXiv.1712.09665

  38. Cheng Z, Liang J, Choi H, Tao G, Cao Z, Liu D, Zhang X (2022) Physical attack on monocular depth estimation with optimal adversarial patches, pp 514–532 . https://doi.org/10.1007/978-3-031-19839-7_30

Download references

Funding

No external funding was received for this research.

Author information

Authors and Affiliations

Authors

Contributions

All authors contributed equally to the research and manuscript preparation. Each author played a significant role in the design of the study, data collection, data analysis, and manuscript writing.

Corresponding author

Correspondence to Rangachary Kommanduri.

Ethics declarations

Consent to participate

All participants provided informed consent to participate in the study.

Consent for publication

All authors consent to the publication of this manuscript.

Ethics approval

This research study was conducted in compliance with all relevant ethical standards and guidelines.

Conflict of Interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kommanduri, R., Ghorai, M. Anomaly detection in video surveillance: a supervised inception encoder approach. Multimed Tools Appl 83, 78517–78534 (2024). https://doi.org/10.1007/s11042-024-18604-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-024-18604-2

Keywords