Skip to main content
SpringerLink
Log in

Analysis of anomaly detection in surveillance video: recent trends and future vision

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Video Surveillance (VS) systems are popular. For enhancing the safety of public lives as well as assets, it is utilized in public places like marketplaces, hospitals, streets, education institutions, banks, shopping malls, city administrative offices, together with smart cities. The main purpose of security applications is the well-timed and also accurate detection of video anomalies. Anomalous activities along with anomalous entities are the video anomalies, which are stated as the irregular or abnormal patterns on the video that doesn’t match the normal trained patterns. Automatic detection of Anomalous activities, say traffic rule infringements, riots, fighting, and stampede in addition to anomalous entities, say, weapons at the sensitive place together with deserted luggage ought to be done. The Anomaly Detection (AD) in VS is reviewed in the paper. This survey concentrates on the Deep Learning (DL) application in finding the exact count, involved individuals and the occurred activity on a larger crowd at every climate condition. The fundamental DL implementation technology concerned in disparate crowd Video Analysis (VA) is discussed. Moreover, it presented the available datasets as well as metrics for performance evaluation and also described the examples of prevailing VS systems utilized in the real life. Lastly, the challenges together with propitious directions for additional research are outlined. Pattern recognition has been the subject of a great deal of study during the previous half-century. There isn’t a single technique that can be utilised for all kinds of applications, whether in bioinformatics or data mining or speech recognition or remote sensing or multimedia or text detection or localization or any other area. Methodologies for object recognition are the primary focus of this paper. All aspects of object recognition, including local and global feature-based algorithms, as well as various pattern-recognition approaches, are examined here. Please note that we have attempted to describe the findings of many technologies and the future extent of this paper’s particular technique. We used the datasets’ properties and other evaluation parameters found in an easily accessible web database. Research in pattern recognition and object recognition can greatly benefit from this study, which identifies the research gaps and limits in this subject.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

References

  1. Al Farid F, Hashim N, Abdullah J, Bhuiyan MR, Shahida Mohd Isa WN, Uddin J, Haque MA, Husen MN (2022) A Structured and Methodological Review on Vision-Based Hand Gesture Recognition System. J Imaging 8:153. https://doi.org/10.3390/jimaging8060153

    Article  Google Scholar 

  2. Barz B, Rodner E, Garcia YG, Denzler J (2018) Detecting regions of maximal divergence for spatio-temporal anomaly detection. IEEE Trans Pattern Anal Mach Intell 41(5):1088–1101

    Google Scholar 

  3. Bozcan I, Le Fevre J, Pham HX, Kayacan E (2021) GridNet image-agnostic conditional anomaly detection for indoor surveillance. IEEE Robot Autom Lett 6(2):1638–1645

    Google Scholar 

  4. Chandrakar R, Raja R, Miri R, Patra RK, Sinha U (2021) Computer Succored Vaticination of Multi-Object Detection and Histogram Enhancement in Low Vision. Int J of Biometrics. Special Issue: Investigation of Robustness in Image Enhancement and Preprocessing Techniques for Biometrics and Computer Vision Applications 1:1–20

  5. Chandrakar R, Raja R, Miri R (2021) Animal detection based on deep convolutional neural networks with genetic segmentation. Multimed Tools Appl. https://doi.org/10.1007/s11042-021-11290-4

  6. Chandrakar R, Miri R, Kushwaha A (2022) Enhanced the moving object detection and object tracking for traffic surveillance using RBF-FDLNN and CBF algorithm. Expert Syst Appl 191:116306, ISSN: 0957-4174. https://doi.org/10.1016/j.eswa.2021.116306

    Article  Google Scholar 

  7. Chang Y, Zhigang T, Xie W, Luo B, Zhang S, Sui H, Yuan J (2022) Video anomaly detection with spatio-temporal dissociation. Pattern Recogn 122:1–12

    Google Scholar 

  8. Chen C, Yu S, Bi X (2015) Detection of anomalous crowd behavior based on the acceleration feature. IEEE Sensors J 15(12):7252–7261

    Google Scholar 

  9. Chen D, Wang P, Yue L, Zhang Y, Jia T (2020) Anomaly detection in surveillance video based on bidirectional prediction. Image Vis Comput 98:1–8

    Google Scholar 

  10. Choudhary S, Lakhwani K, Kumar S (2022) Three Dimensional Objects Recognition & Pattern Recognition Technique; related challenges: a review. Multimed Tool Appl 23(1):1–44

    Google Scholar 

  11. Chu W, Xue H, Yao C, Deng C (2017) Sparse coding guided spatiotemporal feature learning for abnormal event detection in large videos. J Latex Class Files 14(8):1–11

    Google Scholar 

  12. Direkoglu C (2020) Abnormal crowd behavior detection using motion information images and convolutional neural networks. IEEE Access 8:80409–80416

    Google Scholar 

  13. Dong L, Zhang Y, Wen C, Wu H (2016) Camera anomaly detection based on morphological analysis and deep learning. IEEE International Conference on Digital Signal Processing (DSP), 16–18 Oct, Beijing, China

  14. dos Santos FP, Ribeiro LSF, Ponti MA (2019) Generalization of feature embeddings transferred from different video anomaly detection domains. J Vis Commun Image Represent. https://doi.org/10.1016/j.jvcir.2019.02.035

  15. Fadl S, Han Q, Li Q (2020) CNN spatiotemporal features and fusion for surveillance video forgery detection. Signal Process Image Commun 90:116066. https://doi.org/10.1016/j.image.2020.116066

    Article  Google Scholar 

  16. Fan Y, Wen G, Li D, Qiu S, Levine MD, Xiao F (2020) Video anomaly detection and localization via gaussian mixture fully convolutional variational autoencoder. Comput Vis Image Underst 195:1–12

    Google Scholar 

  17. Feng Y, Yuan Y, Lu X (2016) Learning deep event models for crowd anomaly detection. Neurocomputing 219:548–556

    Google Scholar 

  18. Ganokratanaa T, Aramvith S, Sebe N (2020) Unsupervised anomaly detection and localization based on deep spatiotemporal translation network. IEEE Access 8:50312–50329

    Google Scholar 

  19. Gao X, Xu G, Li S, Wu Y, Dancigs E, Juan D (2019) Particle filter-based prediction for anomaly detection in automatic surveillance. IEEE Access 7:107550–107559

    Google Scholar 

  20. Gupta D, Varshney N, Kumar A (2022) Detection of behavioral patterns employing a hybrid approach of computational techniques. Comput Mater Contin 72(1):2015–2031

    Google Scholar 

  21. Hao Y, Li J, Wang N, Wang X, Gao X (2022) Spatiotemporal consistency enhanced network for video anomaly detection. Pattern Recogn 121:1–11

    Google Scholar 

  22. Hu Z-p, Zhang L, Li S-f, Sun D-g (2020) Parallel spatial-temporal convolutional neural networks for anomaly detection and location in crowded scenes. J Vis Commun Image Represent 67:102765. https://doi.org/10.1016/j.jvcir.2020.102765

    Article  Google Scholar 

  23. Khaleghi A, Moin MS (2018) Improved anomaly detection in surveillance videos based on a deep learning method. 8th Conference of AI & Robotics and 10th RoboCup Iranopen International Symposium (IRANOPEN), 10–10 April, Qazvin, Iran

  24. Laxhammar R, Falkman G (2014) Online learning and sequential anomaly detection in trajectories. IEEE Trans Pattern Anal Mach Intell 36(6):1158–1173

    MATH  Google Scholar 

  25. Leyva R, Sanchez V, Li C-T (2015) Video anomaly detection with compact feature sets for online performance. Journal of Latex Class Files 14(8):1–16

    MATH  Google Scholar 

  26. Li N, Chang F (2019) Video anomaly detection and localization via multivariate Gaussian fully convolution adversarial autoencoder. Neurocomputing 369:92–105. https://doi.org/10.1016/j.neucom.2019.08.044

    Article  Google Scholar 

  27. Li Q, Li W (2016) A novel framework for anomaly detection in video surveillance using multi-feature extraction. 9th international symposium on computational intelligence and design, 10–11 Dec, Hangzhou, China

  28. Li Y, Guo T, Xia R, Xie W (2018) Road traffic anomaly detection based on fuzzy theory. IEEE Access 6:40281–40288

    Google Scholar 

  29. Li Y, Cai Y, Liu J, Lang S, Zhang X (2019) Spatio-temporal unity networking for video anomaly detection. IEEE Access 7:172425–172432

    Google Scholar 

  30. Li Z, Li Y, Gao Z (2020) Spatiotemporal representation learning for video anomaly detection. IEEE Access 8:25531–25542

    Google Scholar 

  31. Li A, Miao Z, Cen Y, Zhang X-P, Zhang L, Chen S (2020) Abnormal event detection in surveillance videos based on low-rank and compact coefficient dictionary learning. Pattern Recogn 108:1–16

    Google Scholar 

  32. Li T, Chen X, Zhu F, Zhang Z, Yan H (2021) Two-stream deep spatial-temporal auto-encoder for surveillance video abnormal event detection. Neurocomputing 439:256–270

    Google Scholar 

  33. Li B, Leroux S, Simoens P (2021) Decoupled appearance and motion learning for efficient anomaly detection in surveillance video. Comput Vis Image Underst 210(4):1–8

    Google Scholar 

  34. Lim JY, Al Jobayer MI, Baskaran VM, Lim JMY, See J, Wong KS (2021) Deep multi-level feature pyramids application for non-canonical firearm detection in video surveillance. Eng Appl Artif Intell 97:1–18

    Google Scholar 

  35. Liu C, Wang G, Ning W, Lin X, Li L, Liu Z (2010) Anomaly detection in surveillance video using motion direction statistics. IEEE 17th international conference on image processing, September 26–29, Hong Kong

  36. Liu Y, Yu H, Gong C, Chen Y (2020) A real time expert system for anomaly detection of aerators based on computer vision and surveillance cameras. J Vis Commun Image Represent 68:102767. https://doi.org/10.1016/j.jvcir.2020.102767

    Article  Google Scholar 

  37. Luo W, Liu W, Lian D, Tang J, Duan L, Peng X, Gao S (2019) Video anomaly detection with sparse coding inspired deep neural networks. IEEE Trans Pattern Anal Mach Intell 43(3):1070–1084

    Google Scholar 

  38. Luo W, Liu W, Gao S (2021) Normal graph spatial temporal graph convolutional networks based prediction network for skeleton based video anomaly detection. Neurocomputing 444:332–337

    Google Scholar 

  39. Mansour RF, Gutierrez JE, Gamarra M, Villanueva JA, Leal N (2021) Intelligent video anomaly detection and classification using faster RCNN with deep reinforcement learning model. Image Vis Comput 112:1–9

    Google Scholar 

  40. Maqsood R, Bajwa UI, Saleem G, Raza RH, Anwar MW (2021) Anomaly recognition from surveillance videos using 3D convolution neural network. Multimed Tools Appl 80:18693–18716

    Google Scholar 

  41. Murugesan M, Thilagamani Dr S (2020) Efficient anomaly detection in surveillance videos based on multi layer perception recurrent neural network. Microprocess Microsyst 79:103303. https://doi.org/10.1016/j.micpro.2020.103303

    Article  Google Scholar 

  42. Nawaratne R, Alahakoon D, De Silva D, Yu X (2019) Spatiotemporal anomaly detection using deep learning for real-time video surveillance. IEEE Trans Industr Inform 16(1):393–402

    Google Scholar 

  43. Ovhal KB, Patange SS, Shinde RS, Tarange VK, Kotkar VA (2017) Analysis of anomaly detection techniques in video surveillance. International conference on intelligent sustainable systems, 7–8 Dec, Palladam, India

  44. Pandey S, Miri R, Sinha GR (2022) AFD Filter and E2n2 Classifier for Improving Visualization of Crop Image and Crop Classification in Remote Sensing Image. Int J Remote Sens 43(1). https://doi.org/10.1080/01431161.2021.2000062

  45. Piciarelli C, Micheloni C, Foresti GL (2008) Trajectory-based anomalous event detection. IEEE Trans Circuits Syst Video Technol 18(11):1544–1554

    Google Scholar 

  46. Pramanik A, Sarkar S, Maiti J (2021) A real-time video surveillance system for traffic pre-events detection. Accid Anal Prev 154:1–21

    Google Scholar 

  47. Ragedhaksha, Darshini, Shahil, Nehru A (2021) Deep learning-based real-world object detection and improved anomaly detection for surveillance videos. Mater Today Proc. https://doi.org/10.1016/j.matpr.2021.07.064

  48. Reccetti M, Marfia G, Zanichelli M (2010 Article No.: 28) The art and craft of making the Tortellini: playing with a digital gesture recognizer for preparing pasta culinary recipes. Comput Entertain 8(4):1–20. https://doi.org/10.1145/1921141.1921148

  49. Sahu AK, Sharma S, Tanveer M, Raja R (2021) Internet of Things attack detection using hybrid Deep Learning Model. Comput Commun 176:146–154, ISSN 0140–3664. https://doi.org/10.1016/j.comcom.2021.05.024

    Article  Google Scholar 

  50. SanMiguel JC, Martínez JM, Caro-Campos L (2017) Object-size invariant anomaly detection in videosurveillance. International Carnahan Conference on Security Technology (ICCST), 23–26 Oct, Madrid, Spain

  51. Singh H, Hand EM, Alexis K (2020) Anomalous motion detection on highway using deep learning. ArXiv: 200608143

  52. Sodemann AA, Ross MP, Borghetti BJ (2012) A review of anomaly detection in automated surveillance. IEEE Trans Syst Man Cybern Part C Appl Rev 42(6):1257–1272

    Google Scholar 

  53. Sun J, Wang X, Xiong N, Shao J (2016) Learning sparse representation with variational auto-encoder for anomaly detection. IEEE Access 4:1–10

    Google Scholar 

  54. Tan X, Zhang C, Zha H (2015) Learning to detect anomalies in surveillance video. IEEE Signal Process Lett 22(9):1477–1481

    Google Scholar 

  55. Thomaz LA, Jardim E, da Silva AF, da Silva EAB, Netto SL, Krim H (2017) Anomaly detection in moving-camera video sequences using principal subspace analysis. IEEE Trans Circuits Syst 65(3):1003–1015

    MathSciNet  MATH  Google Scholar 

  56. Tiwari L, Raja R, Awasthi V, Miri R, Sinha GR, Alkinani MH, Polat K (2021) Detection of lung nodule and cancer using novel Mask-3 FCM and TWEDLNN algorithms. Measurement 172:108882, ISSN 0263-2241. https://doi.org/10.1016/j.measurement.2020.108882

    Article  Google Scholar 

  57. Ullah A, Ahmad J, Muhammad K, Sajjad M, Baik SW (2017) Action recognition in video sequences using deep bi-directional LSTM with CNN features. IEEE Access 6:1155–1166

    Google Scholar 

  58. Wang S, Zhu E, Yin J, Porikli F (2017) Video anomaly detection and localization by local motion based joint video representation and OCELM. Neurocomputing 277:161–175. https://doi.org/10.1016/j.neucom.2016.08.156

    Article  Google Scholar 

  59. Wang T, Qiao M, Lin Z, Li C, Snoussi H, Liu Z, Choi C (2018) Generative neural networks for anomaly detection in crowded scenes. J Latex Class Files 14(8):1–11

    Google Scholar 

  60. Wu P, Liu J, Shen F (2019) A deep one-class neural network for anomalous event detection in complex scenes. IEEE Trans Neural Netw Learn Syst 31(7):2609–2622

    Google Scholar 

  61. Wu P, Liu J, Li M, Sun Y, Shen F (2020) Fast sparse coding networks for anomaly detection in videos. Pattern Recogn 107:107515. https://doi.org/10.1016/j.patcog.2020.107515

    Article  Google Scholar 

  62. Wu R, Li S, Chen C, Hao A (2021) Improving video anomaly detection performance by mining useful data from unseen video frames. Neurocomputing 462:523–533

    Google Scholar 

  63. Xiang T, Gong S (2008) Video behavior profiling for anomaly detection. IEEE Trans Pattern Anal Mach Intell 30(5):893–908

    MathSciNet  Google Scholar 

  64. Xie S, Zhang X, Cai J (2018) Video crowd detection and abnormal behavior model detection based on machine learning method. Neural Comput & Applic 31:175–184. https://doi.org/10.1007/s00521-018-3692-x

    Article  Google Scholar 

  65. Xu D, Yan Y, Ricci E, Sebe N (2016) Detecting anomalous events in videos by learning deep representations of appearance and motion. Comput Vis Image Underst 156:117–127. https://doi.org/10.1016/j.cviu.2016.10.010

    Article  Google Scholar 

  66. Xu K, Sun T, Jiang X (2019) Video anomaly detection and localization based on an adaptive intra-frame classification network. IEEE Trans Multimedia 22(2):394–406

    Google Scholar 

  67. Yuan Y, Wang D, Wang Q (2016) Anomaly detection in traffic scenes via spatial-aware motion reconstruction. IEEE Trans Intell Transp Syst 18:1198–1209. https://doi.org/10.1109/TITS.2016.2601655

    Article  Google Scholar 

  68. Zaheer MZ, Mahmood A, Shin H, Lee S-I (2020) A self-reasoning framework for anomaly detection using video-level labels. IEEE Signal Process Lett 27:1705–1709

    Google Scholar 

  69. Zahid Y, Tahir MA, Durrani NM, Bouridane A (2020) IBaggedFCNet an ensemble framework for anomaly detection in surveillance videos. IEEE Access 8:220620–220630

    Google Scholar 

  70. Zaidi S, Jagadeesh B, Sudheesh KV, Audre Arlene A (2017) Video anomaly detection and classification for human activity recognition. International Conference on Current Trends in Computer, Electrical, Electronics and Communication, 8–9 Sept, Mysore, India

  71. Zaigham Zaheer M, Lee JH, Lee S-Ik, Seo B-S (2019) A brief survey on contemporary methods for anomaly detection in videos. International Conference on Information and Communication Technology Convergence (ICTC), 16–18 Oct, Jeju, Korea (South)

  72. Zang X, Li G, Li Z, Li N, Wang W (2016) An object-aware anomaly detection and localization in surveillance videos. IEEE second international conference on multimedia big data, 20–22 April, Taipei, Taiwan

  73. Zhang X, Yang S, Zhang J, Zhang W (2020) Video anomaly detection and localization using motion-field shape description and homogeneity testing. Pattern Recogn 105:1–13

    Google Scholar 

  74. Zhang X, Zheng Y, Zhao Z, Liu Y, Blumenstein M, Li J (2021) Deep learning detection of anomalous patterns from bus trajectories for traffic insight analysis. Knowl-Based Syst 217:1–13

    Google Scholar 

  75. Zhang X, Jie M, Zhang X, Liu H, Zong L, Li Y (2022) Deep anomaly detection with self-supervised learning and adversarial training. Pattern Recogn 121:1–14

    Google Scholar 

  76. Zhou F, Lin W, Li Z, Zuo W, Tan H (2019) Unsupervised learning approach for abnormal event detection in surveillance video by hybrid autoencoder. Neural Process Lett 52:961–975. https://doi.org/10.1007/s11063-019-10113-w

    Article  Google Scholar 

Download references

Funding

The authors have no financial or proprietary interests in any material discussed in this article.

Author information

Authors and Affiliations

  1. Department of Information Technology, Guru Ghasidas Vishwavidyalaya, Bilaspur (CG), India

    Rohit Raja

  2. Department of Information Technology, Manipal University Jaipur, Jaipur, India

    Prakash Chandra Sharma

  3. Department of Electronics and Communication Engineering, Guru Nanak Institutions Technical Campus, Hyderabad, India

    Md Rashid Mahmood

  4. Department of Computer & Communication Engineering, Manipal University Jaipur, Jaipur, India

    Dinesh Kumar Saini

Authors
  1. Rohit Raja
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Prakash Chandra Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Md Rashid Mahmood
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dinesh Kumar Saini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Prakash Chandra Sharma.

Ethics declarations

Conflicts of interest/competing interests

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Raja, R., Sharma, P.C., Mahmood, M.R. et al. Analysis of anomaly detection in surveillance video: recent trends and future vision. Multimed Tools Appl 82, 12635–12651 (2023). https://doi.org/10.1007/s11042-022-13954-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13954-1

Keywords

Search

Navigation