Skip to main content
SpringerLink
Log in

A multi-stage fusion instance learning method for anomalous event detection in videos

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Anomalous event detection in giant amount surveillance footage in real world is currently an active research area. Variery and rareness of the anomaly events is still a thorny challenge to deal with. In this paper, we propose a multi-stage fusion instance learning method (MFIL) for inferring anomalous event pattern and predicting anomaly appearance in videos. We propose object-aware model and action-aware model to represent regularities of human objects and actions among frames exploiting cascaded deep network models. Furthermore we improve and represent fusion instance learning method for fetching and maximizing anomaly scores via object and action regularities in anomalous sequences from videos. We validate the performance of MFIL on action movie and UCF-Crime respectively, both contain anomalous and violent events. Experimental results demonstrated that MFIL is effective for anomalous event detection in videos gathered from real world.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Pang G, Shen C, Cao L, Van Den Hengel A (2021) Deep learning for anomaly detection: a review. ACM Comput Surv (CSUR) 54(2):1–38

    Article  Google Scholar 

  2. Sultani W, Chen C, Shah M (2018) Real-world anomaly detection in surveillance videos. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6479–6488

  3. Akcay S, Atapour-Abarghouei A, Breckon TP (2018) Ganomaly: semi-supervised anomaly detection via adversarial training. In: Asian conference on computer vision. Springer, pp 622–637

  4. Chalapathy R, Chawla S (2019) Deep learning for anomaly detection: a survey. arXiv preprint arXiv:1901.03407

  5. Majhi S, Das S, Bremond F, Dash R, Sa PK (2021) Weakly-supervised joint anomaly detection and classification. arXiv preprint arXiv:2108.08996

  6. Hasan M, Choi J, Neumann J, Roy-Chowdhury AK, Davis LS (2016) Learning temporal regularity in video sequences. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 733–742

  7. Pang G, Yan C, Shen C, van den Hengel A, Bai X (2020) Self-trained deep ordinal regression for end-to-end video anomaly detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12173–12182

  8. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  9. Wang H, Schmid C (2013) Action recognition with improved trajectories. In Proceedings of the IEEE international conference on computer vision, pp 3551–3558

  10. Simonyan K, Zisserman A (2014) Two-stream convolutional networks for action recognition in videos. arXiv preprint arXiv:1406.2199

  11. Carreira J, Zisserman A (2017) Quo vadis, action recognition? a new model and the kinetics dataset. In: proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 6299–6308

  12. Feichtenhofer C, Pinz A, Zisserman A (2016) Convolutional two-stream network fusion for video action recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1933–1941

  13. Feichtenhofer C, Pinz A, Wildes RP (2017) Spatiotemporal residual networks for video action recognition. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 3476–3484

  14. Donahue J, Anne Hendricks L, Guadarrama S, Rohrbach M, Venugopalan S, Saenko K, Darrell T (2015) Long-term recurrent convolutional networks for visual recognition and description. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2625–2634

  15. Wang L, Xiong Y, Wang Z,  Qiao Y, Lin D, Tang X, Van Gool L (2016) Temporal segment networks: Towards good practices for deep action recognition. In: European conference on computer vision. Springer, pp 20–36

  16. Nievas EB, Suarez OD, García GB, Sukthankar R (2011) Violence detection in video using computer vision techniques. In: International conference on Computer analysis of images and patterns. Springer, pp 332–339

  17. Tenenboim-Chekina L, Rokach L, Shapira B (2013) Ensemble of feature chains for anomaly detection. In: International workshop on multiple classifier systems. Springer, pp 295–306

  18. Lu C, Shi J, Jia J (2013) Abnormal event detection at 150 fps in matlab. In: Proceedings of the IEEE international conference on computer vision, pp 2720–2727

  19. Noto K, Brodley C, Slonim D (2012) Frac: a feature-modeling approach for semi-supervised and unsupervised anomaly detection. Data Min Knowl Disc 25(1):109–133

    Article  Google Scholar 

  20. Morais R, Le V, Tran T, Saha B, Mansour M, Venkatesh S (2019) Learning regularity in skeleton trajectories for anomaly detection in videos. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 11996–12004

  21. Islam Z, Rukonuzzaman M, Ahmed R, Kabir Md, Farazi M, et al (2021) Efficient two-stream network for violence detection using separable convolutional ISTM. arXiv preprint arXiv:2102.10590

  22. Peng W, Liu J, Shen F (2019) A deep one-class neural network for anomalous event detection in complex scenes. IEEE Trans Neural Networks Learn Syst 31(7):2609–2622

    Google Scholar 

Download references

Acknowledgements

This paper is supported by Humanities and Social Sciences Foundation of Chinese Ministry of Education (No. 19YJC760150), National Natural Science Foundation (No. 61402016), the open funding project of State Key Laboratory of Virtual Reality Technology and Systems, Beihang University (No. VRLAB2018A05), and National Key Research and Development Program Project (2020YFC0811004).

Author information

Authors and Affiliations

  1. College of Information Science, North China University of Technology, Beijing, 100144, People’s Republic of China

    Jian Cheng, Fengquan Zhang & Guiling Wang

  2. NARI Group Corporation, No.19 Chengxin Avenue, Nanjing, 211000, People’s Republic of China

    Wancai Zhang

Authors
  1. Jian Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fengquan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guiling Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wancai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fengquan Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cheng, J., Zhang, F., Wang, G. et al. A multi-stage fusion instance learning method for anomalous event detection in videos. Int. J. Mach. Learn. & Cyber. 14, 445–454 (2023). https://doi.org/10.1007/s13042-022-01572-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-022-01572-0

Keywords

Search

Navigation