Skip to main content
SpringerLink
Log in

Video anomaly detection based on scene classification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

As a significant research hotspot in the field of computer vision, video anomaly detection plays an essential role in ensuring public safety. Anomaly detection remains a challenging task given the complex situation in public areas and the large random distribution of crowds. The density of people in the same scene varies greatly due to the instability of the pedestrian volume. Specifically, the characteristics of crowd distribution mainly include low density, small aggregation and dispersion, or large aggregation and severe occlusion. Considering the large difference between high-density and low-density crowd characteristics, we propose an anomaly detection algorithm based on scene classification in order to obtain better anomaly detection result. Firstly, we propose a novel scene classification method, which uses pre-trained YoloV4 model to detect the number of people in the video frames and generate heatmaps, and extracts pixel features through the Double-Canny algorithm to represent the occlusion degree of the crowd. Furthermore, K-Means clustering is used to adaptively divide the scene into sparse and dense. Secondly, the Generative Adversarial Network (GAN) based on prediction and reconstruction is introduced to detect anomalies respectively, and the final accuracy is achieved by combining the detection accuracy of both networks. Finally, experiments on three benchmark datasets demonstrate the competitive performance of our method with the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

We provide original and editable data appearing in the submitted article, including figures, tables and experimental results.

Code availability

We are pleased to share code that is used in work submitted for publication. Authors' contributions: All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Hongjun Li, Xulin Shen, Xiaohu Sun, Yunlong Wang, Chaobo Li, Junjie Chen. The first draft of the manuscript was written by Xulin Shen and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

References

  1. Abati D, Porrello A, Calderara S et al (2019) Latent space autoregression for novelty detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 481–490. IEEE Computer Society, Long Beach, USA

  2. Alafif Tarik et al (2022) Generative adversarial network based abnormal behavior detection in massive crowd videos: a hajj case study. J Ambient Intell Humaniz Comput 13(8):4077–4088

    Article  Google Scholar 

  3. Alanazi AA, Bilal M (2019) Crowd density estimation using novel feature descriptor. arXiv preprint arXiv:1905.05891

  4. Bhuiyan MR, Abdullah J, Hashim N et al (2022) A deep crowd density classification model for Hajj pilgrimage using fully convolutional neural network. PeerJ Comput Sci 8:e895

    Article  Google Scholar 

  5. Bhuiyan R, Abdullah J, Hashim N et al (2022) Deep dilated convolutional neural network for crowd density image classification with dataset augmentation for hajj pilgrimage. Sensors 22(14):5102

    Article  Google Scholar 

  6. Bochkovskiy A, Wang CY, Liao HYM (2020) Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934

  7. Chen D, Yue L, Chang X et al (2021) NM-GAN: Noise-modulated generative adversarial network for video anomaly detection. Pattern Recognit 116:107969

    Article  Google Scholar 

  8. Gong S, Bourennane EB (2019) A method based on texture feature and edge detection for people counting in a crowded area. In: Digital Image and Signal Processing

  9. Gong D, Liu L, Le V et al (2019) Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1705–1714. Institute of Electrical and Electronics Engineers Inc, Seoul, Korea

  10. Goodfellow IJ, Pouget-Abadie J, Mirza M et al (2014) Generative adversarial networks. Adv Neural Inf Process Syst 3:2672–2680

    Google Scholar 

  11. Hussain N, Yatim HSM, Hussain NL et al (2011) CDES: A pixel-based crowd density estimation system for Masjid al-Haram. Saf Sci 49(6):824–833

    Article  Google Scholar 

  12. Huynh VS, Tran VH, Huang CC (2019) Iuml: Inception u-net based multi-task learning for density level classification and crowd density estimation. In: IEEE International Conference on Systems, Man and Cybernetics (SMC), pp 3019–3024. IEEE

  13. Jia D, Zhang C, Zhang B (2021) Crowd density classification method based on pixels and texture features. Mach Vis Appl 32(2):1–22

    Article  Google Scholar 

  14. Jiang X, Zhang L, Xu M et al (2020) Attention scaling for crowd counting. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 4705–4714. IEEE Computer Society, Seattle, USA

  15. Lamba S, Nain N (2017) A large scale crowd density classification using spatio-temporal local binary pattern. In: 13th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), pp 296–302. IEEE

  16. Lamba S, Nain N (2019) A texture based mani-fold approach for crowd density estimation using Gaussian Markov Random Field. Multimedia Tools Apply 78(5):5645–5664

    Article  Google Scholar 

  17. Lazaridis L, Dimou A, Daras P (2018) Abnormal behavior detection in crowded scenes using density heatmaps and optical flow. In: 26th European Signal Processing Conference (EUSIPCO), pp 2060–2064. European Signal Processing Conference, Rome

  18. Lee S, Kim HG, Ro YM (2020) BMAN: bidirectional multi-scale aggregation networks for abnormal event detection. IEEE Trans Image Process 29:2395–2408

    Article  MATH  Google Scholar 

  19. Lei Z, Deng F, Yang X (2019) Spatial temporal balanced generative adversarial autoencoder for anomaly detection. In: Proceedings of the 2019 International Conference on Image, Video and Signal Processing, pp 1–7

  20. Li N, Chang F, Liu C (2020) Spatial-temporal cascade autoencoder for video anomaly detection in crowded scenes. IEEE Trans Multimed 23:203–215

    Article  Google Scholar 

  21. Liu W, Luo W, Lian D et al (2018) Future frame prediction for anomaly detection - a new baseline. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 6536–6545. IEEE Computer Society, Salt Lake City

  22. Marsden M, McGuinness K, Little S et al (2017) Resnetcrowd: A residual deep learning architecture for crowd counting, violent behaviour detection and crowd density level classification. In: 14th IEEE international conference on advanced video and signal based surveillance (AVSS), pp 1–7. IEEE

  23. Naeem H, Cheng X, Ullah F et al (2022) A deep convolutional neural network stacked ensemble for malware threat classification in internet of things. J Circuits Syst Comput 31:2250302

    Article  Google Scholar 

  24. Ravanbakhsh M, Sangineto E, Nabi M et al (2019) Training adversarial discriminators for cross-channel abnormal event detection in crowds. In: 19th IEEE Winter Conference on Applications of Computer Vision, pp 1896–1904. Institute of Electrical and Electronics Engineers Inc, Hilton Waikoloa Village

  25. Sabokrou M, Khalooei M, Fathy M et al (2018) Adversarially learned one-class classifier for novelty detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3379–3388. IEEE Computer Society, Salt Lake City, USA

  26. Samriya JK, Tiwari R, Cheng X et al (2022) Network intrusion detection using ACO-DNN model with DVFS based energy optimization in cloud framework. Sustain Comput Inform Syst 35:100746

    Google Scholar 

  27. Song H, Sun C, Wu X et al (2019) Learning normal patterns via adversarial attention-based autoencoder for abnormal event detection in videos. IEEE Trans Multimedia 22(8):2138–2148

    Article  Google Scholar 

  28. Wang P, Wang P, Fan E (2021) Violence detection and face recognition based on deep learning. Pattern Recogn Lett 142:20–24

    Article  Google Scholar 

  29. Xiong G, Cheng J, Wu X et al (2012) An energy model approach to people counting for abnormal crowd behavior detection. Neurocomputing 83:121–135

    Article  Google Scholar 

  30. Xu M, Ge Z, Jiang X et al (2019) Depth information guided crowd counting for complex crowd scenes. Pattern Recogn Lett 125:563–569

    Article  Google Scholar 

  31. Zhu L, Li C, Yang Z et al (2020) Crowd density estimation based on classification activation map and patch density level. Neural Comput Appl 32(9):5105–5116

    Article  Google Scholar 

Download references

Funding

This work is supported in part by National Natural Science Foundation of China under Grant 61871241, Grant 61971245 and Grant 61976120, in part by Jiangsu Industry University Research Cooperation Project BY2021349, in part by Nantong Science and Technology Program JC2021131 and in part by Postgraduate Research and Practice Innovation Program of Jiangsu Province KYCX21_3084 and KYCX22_3340.

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Nantong University, 9 Seyuan Road, Nantong, 226019, Jiangsu Province, People’s Republic of China

    Hongjun Li, Xulin Shen, Xiaohu Sun, Yunlong Wang, Chaobo Li & Junjie Chen

  2. Nantong Research Institute for Advanced Communication Technologies, Nantong, 226019, Jiangsu, People’s Republic of China

    Hongjun Li & Junjie Chen

Authors
  1. Hongjun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xulin Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaohu Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yunlong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chaobo Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Junjie Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongjun Li.

Ethics declarations

Conflicts of interest/Competing interests

None.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, H., Shen, X., Sun, X. et al. Video anomaly detection based on scene classification. Multimed Tools Appl 82, 45345–45365 (2023). https://doi.org/10.1007/s11042-023-15328-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15328-7

Keywords

Search

Navigation