Skip to main content

Advertisement

SpringerLink
Log in

Real time violence detection in surveillance videos using Convolutional Neural Networks

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Real-time violence detection with the use of surveillance is the process of using live videos to detect violent and irregular behavior. In organizations, they use some potential procedures for recognition the activity in which normal and abnormal activities can be found easily. In this research, multiple key challenges have been oncorporated with the existing work and the proposed work contrast. Firstly, violent objects can’t be defined manually and then the system needs to deal with the uncertainty. The second step is the availability of label dataset because manually annotation video is an expensive and labor-intensive task. There is no such approach for violence detection with low computation and high accuracy in surveillance environments so far. The Convolutional Neural Network’s (CNN) models have been evaluated with the proposed MobileNet model. The MobileNet model has been contrasted with AlexNet, VGG-16, and GoogleNet models. The simulations have been executed using Python from which the accuracy of AlexNet is 88.99 and the loss is 2.480 (%). The accuracy of VGG-16 is 96.49 and loss is 0.1669, the accuracy of GoogleNet is 94.99 and loss is 2.92416 (%). The proposed MobileNet model accuracy is 96.66 and loss is 0.1329 (%). The proposed MobileNet model has shown outstanding performance in the perspective of accuracy, loss, and computation time on the hockey fight dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Ajani OS, El-Hussieny H (2019) An ANFIS-based Human Activity Recognition using IMU sensor Fusion. In: 2019 Novel Intelligent and Leading Emerging Sciences Conference (NILES), pp 34–37

  2. Afza F, Khan MA, Sharif M, Kadry S, Manogaran G, Saba T et al (2021) A framework of human action recognition using length control features fusion and weighted entropy-variances based feature selection. Image Vis Comput 106:104090

    Article  Google Scholar 

  3. Ciechanowski L, Przegalinska A, Magnuski M, Gloor P (2019) In the shades of the uncanny valley: An experimental study of human–chatbot interaction. Futur Gener Comput Syst 92:539–548

    Article  Google Scholar 

  4. Ha J, Park J, Kim H, Park H, Paik J (2018) Violence detection for video surveillance system using irregular motion information. In: International Conference on Electronics, Information, and Communication (ICEIC), 2018, pp 1–3

  5. Halder R, Chatterjee R (2020) CNN-BiLSTM model for violence detection in smart surveillance. SN Comput Sci 1:1–9

    Article  Google Scholar 

  6. Hu J, Liao X, Wang W, Qin Z (2022) Detecting compressed deepfake videos in social networks using frame-temporality two-stream convolutional network. In: IEEE Transactions on Circuits and Systems for Video Technology 32(3):1089–1102. https://doi.org/10.1109/TCSVT.2021.3074259

  7. Jalal A, Mahmood M, Hasan AS (2019) Multi-features descriptors for human activity tracking and recognition in Indoor-outdoor environments. In: 2019 16th International Bhurban Conference on Applied Sciences and Technology (IBCAST), pp 371–376

  8. Jeeva S, Sivabalakrishnan M (2019) Twin background model for foreground detection in video sequence. Cluster Comput 22:11659–11668

    Article  Google Scholar 

  9. Juba B, Le HS (2019) Precision-recall versus accuracy and the role of large data sets. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 4039–4048

  10. Karumuri S, Niewiadomski R, Volpe G, Camurri A (2019) From motions to emotions: classification of affect from dance movements using deep learning. In: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, 2019, pp 1–6

  11. Kiran S, Khan MA, Javed MY, Alhaisoni M, Tariq U, Nam Y et al (2021) Multi-layered deep learning features fusion for human action recognition. Computers, Materials & Continua 69(3):4061–4075. https://doi.org/10.32604/cmc.2021.017800

  12. Khalid M, Keming M, Hussain T (2021) Design and implementation of clothing fashion style recommendation system using deep learning. Rom J Inform Technol Autom Control 31(4):123–136. ISSN 1220–1758. https://doi.org/10.33436/v31i4y202110

  13. Lawal IA, Bano S (2019) Deep human activity recognition using wearable sensors. In: Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments, pp 45–48

  14. Liao X, Yu Y, Li B, Li Z, Qin Z (2019) A new payload partition strategy in color image steganography. IEEE Trans Circ Syst Video Technol 30:685–696

    Article  Google Scholar 

  15. Liao X, Li K, Zhu X, Liu KR (2020) Robust detection of image operator chain with two-stream convolutional neural network,. IEEE J Selec Topics Signal Process 14:955–968

    Article  Google Scholar 

  16. Liu L, Zheng Y, Tang D, Yuan Y, Fan C, Zhou K (2019) NeuroSkinning: Automatic skin binding for production characters with deep graph networks. ACM Trans Graphics (TOG) 38:1–12

    Google Scholar 

  17. Ma J, Ma Y, Li C (2019) Infrared and visible image fusion methods and applications: A survey. Inform Fusion 45:153–178

    Article  Google Scholar 

  18. Muhammad K, Khan S, Palade V, Mehmood I, De Albuquerque VHC (2019) Edge intelligence-assisted smoke detection in foggy surveillance environments,. IEEE Trans Ind Inf 16:1067–1075

    Article  Google Scholar 

  19. Nweke HF, Teh YW, Mujtaba G, Al-Garadi MA (2019) Data fusion and multiple classifier systems for human activity detection and health monitoring: Review and open research directions. Inform Fusion 46:147–170

    Article  Google Scholar 

  20. Ogawa R, Nishikawa J, Hideura E, Goto A, Koto Y, Ito S et al (2019) Objective assessment of the utility of chromoendoscopy with a support vector machine. J Gastrointest Cancer 50:386–391

    Article  Google Scholar 

  21. Pansuriya P, Chokshi N, Patel D, Vahora S (2020) Human activity recognition with event-based dynamic vision sensor using deep recurrent neural network. Int J Adv Sci Technol 29(4):9084–9091

  22. Sezer S, Surer E (2019) Information augmentation for human activity recognition and fall detection using empirical mode decomposition on smartphone data. In: Proceedings of the 6th International Conference on Movement and Computing, pp 1–8

  23. Siddiqi MH, Alruwaili M, Ali A (2019) A novel feature selection method for video-based human activity recognition systems. IEEE Access 7:119593–119602

    Article  Google Scholar 

  24. Singh T, Vishwakarma DK (2019) Human activity recognition in video benchmarks: A survey. Advances in Signal Processing and Communication (ed). Springer, Berlin, pp 247–259

  25. Singh R, Kushwaha AKS, Srivastava R (2019) Multi-view recognition system for human activity based on multiple features for video surveillance system. Multimed Tools Appl 78:17165–17196

    Article  Google Scholar 

  26. Sobral A, Vacavant A (2014) A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos. Comput Vis Image Underst 122:4–21

    Article  Google Scholar 

  27. Subedar M, Krishnan R, Meyer PL, Tickoo O, Huang J (2019) Uncertainty-aware audiovisual activity recognition using deep bayesian variational inference. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 6301–6310

  28. Ullah A, Ahmad J, Muhammad K, Sajjad M, Baik SW (2017) Action recognition in video sequences using deep bi-directional LSTM with CNN features. IEEE Access 6:1155–1166

    Article  Google Scholar 

  29. Ullah A, Muhammad K, Haq IU, Baik SW (2019) Action recognition using optimized deep autoencoder and CNN for surveillance data streams of non-stationary environments,. Futur Gener Comput Syst 96:386–397

    Article  Google Scholar 

  30. Ullah W, Ullah A, Haq IU, Muhammad K, Sajjad M, Baik SW (2021) CNN features with bi-directionalLSTM for real-time anomaly detection in surveillance networks. Multimed Tools Appl 80:16979–16995. https://doi.org/10.1007/s11042-020-09406-3

    Article  Google Scholar 

  31. Voicu R-A, Dobre C, Bajenaru L, Ciobanu R-I (2019) Human physical activity recognition using smartphone sensors. Sensors 19:458

  32. Žemgulys J, Raudonis V, Maskeliūnas R, Damaševičius R (2020) Recognition of basketball referee signals from real-time videos. J Ambient Intell Humaniz Comput 11:979–991

    Article  Google Scholar 

  33. Zhao R, Yan R, Chen Z, Mao K, Wang P, Gao RX (2019) Deep learning and its applications to machine health monitoring,. Mech Syst Signal Process 115:213–237

    Article  Google Scholar 

  34. Zhu J, Chen H, Ye W (2020) Classification of human activities based on radar signals using 1D-CNN and LSTM. In: 2020 IEEE International Symposium on Circuits and Systems (ISCAS), pp 1–5

  35. Zhuang Z, Xue Y (2019) Sport-related human activity detection and recognition using a smartwatch. Sensors 19:5001

  36. Zou H, Yang J, Prasanna Das H, Liu H, Zhou Y, Spanos CJ (2019) WiFi and vision multimodal learning for accurate and robust device-free human activity recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp 0–0

Download references

Acknowledgements

This work was supported by the Key Research and Development Program of Zhejiang Province under Grant 2020C01076, and by the National Natural Science Foundation of China under Grant 62172366.

Author information

Authors and Affiliations

  1. Institute of Computer Sciences and Information Technology, The University of Agriculture, Peshawar, Pakistan

    Irfanullah, Arshad Iqbal & Altaf Hussain

  2. School of Computer Science and Information Engineering, Zhejiang Gongshang University, Hangzhou, China

    Tariq Hussain & Bailin Yang

Authors
  1. Irfanullah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tariq Hussain
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Arshad Iqbal
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bailin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Altaf Hussain
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Tariq Hussain: contributed equally and are co-first authors, Writing- Review and Editing, Visualization, Writing-Original Draft. Resources, Validation.: Irfanullah: Methodology, Software Formal analysis.: Binlin Yang: Conceptualization, Investigation, Review, and Editing.: Arshad Iqbal: Supervision.

Corresponding authors

Correspondence to Tariq Hussain or Bailin Yang.

Ethics declarations

Conflict of interest

The authors declared that they have no conflicts of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Irfanullah, Hussain, T., Iqbal, A. et al. Real time violence detection in surveillance videos using Convolutional Neural Networks. Multimed Tools Appl 81, 38151–38173 (2022). https://doi.org/10.1007/s11042-022-13169-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13169-4

Keywords

Search

Navigation