Skip to main content
SpringerLink
Log in

FOAD: a novel video anomaly detection focusing on objects

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With the popularity of surveillance video, people’s increased security awareness and demand for risk warning, video anomaly detection is gradually gaining widespread attention. Since the amount of data in the background is larger than that of the foreground object, the background information of video frames is more easily noticed by the model, while the abnormal behaviors that cannot be easily detected mostly occur in the foreground. At the same time, if the background information is processed separately from the foreground information, it will inevitably increase the complexity of the model. In this paper, we propose a novel video anomaly detection based on scene object. In the training phase, we put foreground objects on the canvas for training and the memory module remembers various normal patterns of the foreground objects, so that the model can be more sensitive to the anomalies in the foreground objects during detection. Our method is able to achieve AUC 74.01% and 30 FPS on ShanghaiTech dataset. At the same time, several advanced video anomaly detection algorithms are compared, which all demonstrate the superiority of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

We provide original and editable data appearing in the submitted article, including figures, tables and experimental results.

Code availability

We are pleased to share code that is used in work submitted for publication.

References

  1. Bahrami M, Pourahmadi M, Vafaei A, Shayesteh MR (2021) A comparative study between single and multi-frame anomaly detection and localization in recorded video streams. J Vis Commun Image Represent 79(1):1–10

    Google Scholar 

  2. Chang, YP, Tu, ZG, Xie, W, Yuan, JS (2020) Clustering driven deep autoencoder for video anomaly detection. In: European Conference on Computer Vision, Glasgow, pp 329–345

  3. Chang YP, Tu ZG, Xie W, Luo B, Zhang SF, Sui HG, Yuan JS (2022) Video anomaly detection with spatio-temporal dissociation. Pattern Recogn 122(1):1–12

    Google Scholar 

  4. Feng, JC, Hong, FT, Zheng, WS (2021) MIST: multiple instance self-training framework for video anomaly detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 14009–14018. Virtual

  5. Gong, D, Liu, LQ, Le, V, Saha, B, Mansour, MR, Venkatesh, S, Hengel, AV (2019) Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection. In: IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, pp 1705–1714

  6. He KM, Zhang XY, Ren SQ Sun (2017) Simple online and Realtime tracking with a deep association metric. In: IEEE International Conference on Image Processing, Beijing, pp 3645–3649

  7. Lee S, Kim HG, Choi DH, Kim H, Ro YM (2021) Video prediction recalling Long-term motion context via memory alignment learning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 3053–3062. Virtual

  8. Li B, Zhang S, Li K(2017)Towards a multi-layers anomaly detection framework for analyzing network traffic. Concurrency and Computation: Practice and Experience 29(14):1-13

    Google Scholar 

  9. Li B, Leroux S, Simoens P (2021) Decoupled appearance and motion learning for efficient anomaly detection in surveillance video. Comput Vis Image Underst 210(9):1–10

    Google Scholar 

  10. Li Q, Yang R, Xiao F, Bhanu B, Zhang F (2022) Attention-based anomaly detection in multi-view surveillance videos. Knowl-Based Syst 252(9):109348

    Article  Google Scholar 

  11. Li DH, Nie XS, Li XF, Zhang Y, Yin YL (2022) Context-related video anomaly detection via generative adversarial network. Pattern Recogn Lett 156(4):183–189

    ADS  Google Scholar 

  12. Li CB, Li HJ, Zhang GA (2023) Future frame prediction based on generative assistant discriminative network for anomaly detection. Appl Intell 53(1):542–559

    Article  Google Scholar 

  13. Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollar P, Zitnick CL (2014) Microsoft COCO: Common objects in context. In: European Conference on Computer Vision, Zurich, pp 740–755

  14. Lin TY, Goyal P, Girshick R, He K, Dollar P (2020) Focal loss for dense object detection. IEEE Trans Pattern Anal Mach Intell 42(2):318–317

    Article  Google Scholar 

  15. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) SSD: Single shot multibox detector. In: European Conference on Computer Vision, Amsterdam, pp 21–37

  16. Liu W, Luo WX, Lian DZ, Gao SH (2018) Future frame prediction for anomaly detection–a new baseline. In: IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, pp 6536–6545

  17. Liu ZA, Nie YW, Long CJ, Zhang Q, Li GQ (2021) A hybrid video anomaly detection framework via memory-augmented flow reconstruction and flow-guided frame prediction. In: International Conference on Computer Vision, Montreal, pp 13588–13597

  18. Luo, WX, Liu, W, Gao, SH (2017) A revisit of sparse coding based anomaly detection in stacked RNN framework. In: International Conference on Computer Vision, Venice, pp 341–349

  19. Lv H, Chen C, Cui Z, Xu CY, Li Y, Yang J (2021) Learning normal dynamics in videos with meta prototype network. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 15420–15429. Virtual

  20. Nazia A, Prateek KR, Maheshkumar HK (2022) A3N: Attention-based adversarial autoencoder network for detecting anomalies in video sequence. J Vis Commun Image Represent 87(8):103598

    Google Scholar 

  21. Olaf R., Philipp F, Thomas B (2015) U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-assisted Intervention, Munich, pp 234–241

  22. Park H, Noh J, Ham B (2020) Learning memory-guided normality for anomaly detection. In: IEEE Conference on Computer Vision and Pattern Recognition, Seattle, pp 14360–14369

  23. Ren SQ, He KM, Girshick R, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  24. Saligrama V, Konrad J, Jodoin P (2010) Video anomaly identification. IEEE Signal Process Mag 27(5):18–33

    Article  ADS  Google Scholar 

  25. Sun P, Zhang RF, Jiang Y, Kong T, Xu CF, Zhan W, Tomizuka M, Li L, Yuan ZH, Wang CH, Luo P (2021) Sparse R-CNN: end-to-end object detection with learnable proposals. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 14449–14458. Virtual

  26. Ultralytics/ yolov5 (2020) Available online: https://github.com/ultralytics/yolov5 (accessed on 25 June 2020)

  27. Wojke N, Bewley A, Paulus D (2017) Simple online and Realtime tracking with a deep association metric. In: IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, pp 3645–3649

  28. Ye M, Peng X, Gan W, Wu W, Qiao Y (2019) Anopcn:Video anomaly detection via deep predictive coding network. In: Proceedings of the 27th ACM International Conference on Multimedia, New York, pp 1805-1813

  29. Zhang Y, Hutchinson P, Lieven NAJ, Nunez-Yanez J (2020) Remaining useful life estimation using long short-term memory neural networks and deep fusion. IEEE Access 8(1):19033–19045

    Article  Google Scholar 

  30. Zhong YH, Chen X, Jiang JY, Ren F (2022) A cascade reconstruction model with generalization ability evaluation for anomaly detection in videos. Pattern Recogn 122(2):1–13

    ADS  Google Scholar 

Download references

Funding

This work is supported in part by National Natural Science Foundation of China under Grant 61871241, Grant 61971245 and Grant 61976120, in part by Jiangsu Industry University Research Cooperation Project BY2021349, in part by Nantong Science and Technology Program JC2021131 and in part by Postgraduate Research and Practice Innovation Program of Jiangsu Province KYCX21_3084 and KYCX22_3340.

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Nantong University, 9 Seyuan road, Nantong, 226019, Jiangsu province, People’s Republic of China

    Hongjun Li, Jinyi Chen, Xiezhou Huang, Yuxing Zhang, Yunlong Du & Junjie Chen

  2. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210023, Jiangsu, People’s Republic of China

    Hongjun Li

  3. Nantong Research Institute for Advanced Communication Technologies, Nantong, 226019, Jiangsu, People’s Republic of China

    Hongjun Li & Junjie Chen

Authors
  1. Hongjun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jinyi Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiezhou Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuxing Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yunlong Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Junjie Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Hongjun Li, Jinyi Chen, Xiezhou Huang, Yuxing Zhang,Yunlong Du and Junjie Chen. The first draft of the manuscript was written by Jinyi Chen and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Hongjun Li.

Ethics declarations

Conflicts of interest/competing interests

None.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, H., Chen, J., Huang, X. et al. FOAD: a novel video anomaly detection focusing on objects. Multimed Tools Appl 83, 20637–20651 (2024). https://doi.org/10.1007/s11042-023-16429-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16429-z

Keywords

Search

Navigation