Skip to main content
Springer Nature Link
Log in

Anomaly detection with multi-scale pyramid grid templates

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper, we propose a method for abnormal event detection in videos based on Multi-scale Pyramid Grid Templates (MPGT). Unlike traditional methods that usually finish anomaly detection based on a single scale feature, we propose to detect anomalies with a designed multi-scale normalized motion feature in the framework of MPGT. In our work, two scene models are proposed, including a global model and an online model, because the anomalies often occur in regions with moving objects. In addition, we propose a fast method for computing high-scale motion features using a convolution operation based on the first scale feature, and design a scheme for combining the detection results at different scales using vote and pyramid strategies. Experiments on public datasets show that our method has a balanced performance on all the testing datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. The results have been retained with two significant digits, for that we obtain the performance from different references, where some of the detection values only have two significant digits.

References

  1. Adam A, Rivlin E, Shimshoni I, Reinitz D (2008) Robust real-time unusual event detection using multiple fixed-location monitors. IEEE Trans Pattern Anal Mach Intell 30(3):555–560

    Article  Google Scholar 

  2. Basharat A, Gritai A, Shah M (2008) Learning object motion patterns for anomaly detection and improved object detection. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, 1–8

  3. Chatfield K, Simonyan K, Vedaldi A, Zisserman A, Valstar MF, French AP, Pridmore TP (2014) Return of the devil in the details: delving deep into convolutional nets. In: Valstar MF, French AP, Pridmore TP (eds) British Machine Vision Conference, BMVC 2014, Nottingham, UK, September 1–5, 2014. BMVA Press

    Google Scholar 

  4. Cong Y, Yuan J, Liu J (2011) Sparse reconstruction cost for abnormal event detection, 3449–3456

  5. Cong Y, Yuan J, Tang Y (2013) Video anomaly search in crowded scenes via spatio-temporal motion context. IEEE Trans Inf Forensics Secur 8(10):1590–1599

    Article  Google Scholar 

  6. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. IEEE Computer Society, 886–893

  7. Dutta JK, Banerjee B, Bonet B, Koenig S (2015) Online detection of abnormal events using incremental coding length. In: Bonet B, Koenig S (eds) Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, January 25-30, 2015, Austin, Texas, USA. AAAI Press, pp 3755–3761

  8. Farnebäck G, Bigün J, Gustavsson T (2003) Two-frame motion estimation based on polynomial expansion. In: Bigün J, Gustavsson T (eds) Image Analysis, 13th Scandinavian Conference, SCIA 2003, Halmstad, Sweden, June 29 - July 2, 2003, Proceedings, vol 2749. Lecture Notes in Computer Science. Springer, pp 363–370

    Google Scholar 

  9. Gandhi T, Trivedi MM (2007) Pedestrian protection systems: issues, survey, and challenges. IEEE Trans Intell Transp Syst 8(3):413–430

    Article  Google Scholar 

  10. Giorno AD, Bagnell JA, Hebert M (2016) A discriminative framework for anomaly detection in large videos. In: Leibe B, Matas J, Sebe N et al (eds) Computer Vision - ECCV 2016–14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part V, vol 9909. Lecture Notes in Computer Science. Springer, pp 334–349

    Google Scholar 

  11. Hasan M, Choi J, Neumann J, Roy-Chowdhury AK, Davis LS (2016) Learning temporal regularity in video sequences. IEEE Computer Society, 733–742

  12. He C, Shao J, Sun J (2018) An anomaly-introduced learning method for abnormal event detection. Multim Tools Appl 77(22):29573–29588

    Article  Google Scholar 

  13. Hu W et al (2006) A system for learning statistical motion patterns. IEEE Trans Pattern Anal Mach Intell 28(9):1450–1464

    Article  Google Scholar 

  14. Hu X, Huang Y, Gao X, Luo L, Duan Q (2019) Squirrel-cage local binary pattern and its application in video anomaly detection. IEEE Trans Inf Forensics Secur 14(4):1007–1022

    Article  Google Scholar 

  15. Hu Y, Zhang Y, Davis LS (2013) Unsupervised abnormal crowd activity detection using semiparametric scan statistic. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR Workshops 2013, Portland, OR, USA, June 23-28, 2013, pp 767–774

  16. Ionescu RT, Smeureanu S, Alexe B, Popescu M (2017) Unmasking the abnormal events in video. 2914–2922

  17. Ionescu RT, Khan FS, Georgescu M, Shao L (2019) Object-centric auto-encoders and dummy anomalies for abnormal event detection in video. Computer Vision Foundation / IEEE, 7842–7851

  18. Johnson N, Hogg DC (1996) Learning the distribution of object trajectories for event recognition. Image Vis Comput 14(8):609–615

    Article  Google Scholar 

  19. Kim J, Grauman K (2009) Observe locally, infer globally: a space-time MRF for detecting abnormal activities with incremental updates. 2921–2928

  20. Kratz L, Nishino K (2009) Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models. 1446–1453

  21. Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. IEEE Computer Society, 2169–2178

  22. Li N, Chang F (2019) Video anomaly detection and localization via multivariate gaussian fully convolution adversarial autoencoder. Neurocomputing 369:92–105

    Article  Google Scholar 

  23. Li N, Guo H, Xu D, Wu X (2014) Multi-scale analysis of contextual information within spatio-temporal video volumes for anomaly detection. IEEE, 2363–2367

  24. Li S, Liu C, Yang Y (2018) Anomaly detection based on maximum a posteriori. Pattern Recognit Lett 107:91–97

    Article  Google Scholar 

  25. Lin T et al (2017) Feature pyramid networks for object detection. IEEE Computer Society, 936–944

  26. Liu W, Luo W, Lian D, Gao S (2018) Future frame prediction for anomaly detection - a new baseline. IEEE Computer Society, 6536–6545

  27. Liu W, Luo W, Li Z, Zhao P, Gao S, Kraus S (2019) Margin learning embedded prediction for video anomaly detection with a few anomalies. In: Kraus S (ed) Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019. ijcai.org, pp 3023–3030

  28. Lu C, Shi J, Jia J (2013) Abnormal event detection at 150 FPS in MATLAB. 2720–2727

  29. Luo W, Liu W, Gao S (2017a) Remembering history with convolutional LSTM for anomaly detection. IEEE Computer Society, 439–444

  30. Luo W, Liu W, Gao S (2017b) A revisit of sparse coding based anomaly detection in stacked RNN framework. IEEE Computer Society, 341–349

  31. Mahadevan V, Li W, Bhalodia V, Vasconcelos N (2010) Anomaly detection in crowded scenes. 1975–1981

  32. Medioni GG, Cohen I, Brémond F, Hongeng S, Nevatia R (2001) Event detection and analysis from video streams. IEEE Trans Pattern Anal Mach Intell 23(8):873–889

    Article  Google Scholar 

  33. Mehran R, Oyama A, Shah M (2009) Abnormal crowd behavior detection using social force model. 935–942

  34. Mo X, Monga V, Bala R, Fan Z (2014) Adaptive sparse representations for video anomaly detection. IEEE Trans Circuits Syst Video Techn 24(4):631–645

    Article  Google Scholar 

  35. Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man and Cybernetics 1:62–66

    Article  Google Scholar 

  36. Nguyen T, Meunier J (2019) Anomaly detection in video sequence with appearance-motion correspondence. IEEE, 1273–1283

  37. Pang G, Shen C, van den Hengel A, Teredesai A et al (2019) Deep anomaly detection with deviation networks. In: Teredesai A, Kumar V, Li Y, et al (eds) Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4-8, 2019. ACM, pp 353–362

  38. Pang G, Yan C, Shen C, van den Hengel A, Bai X (2020) Self-trained deep ordinal regression for end-to-end video anomaly detection. IEEE, 12170–12179

  39. Piciarelli C, Micheloni C, Foresti GL (2008) Trajectory-based anomalous event detection. IEEE Trans Circuits Syst Video Techn 18(11):1544–1554

    Article  Google Scholar 

  40. Reddy V, Sanderson C, Lovell BC (2011) Improved anomaly detection in crowded scenes via cell-based analysis of foreground speed, size and texture. IEEE Computer Society, 55–61

  41. Ronneberger O, Fischer P, Brox T, Navab N, Hornegger J, III WMW, Frangi AF (2015) U-net: convolutional networks for biomedical image segmentation. In: Navab N, Hornegger J, III WMW, Frangi AF (eds) Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015 - 18th International Conference Munich, Germany, October 5 - 9, 2015, Proceedings, Part III, Lecture Notes in Computer Science, vol 9351. Springer, pp 234–241

  42. Shi X, et al (2015) Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In: Cortes C, Lawrence ND, Lee DD, Sugiyama M, Garnett R (eds) Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pp 802–810

  43. Sultani W, Chen C, Shah M (2018) Real-world anomaly detection in surveillance videos. IEEE Computer Society, pp 6479–6488

  44. Tran H, Hogg DC (2017) Anomaly detection using a convolutional winner-take-all autoencoder. In: British Machine Vision Conference 2017, BMVC 2017, London, UK, September 4-7, 2017. BMVA Press

  45. Vu H, Nguyen TD, Le T, et al (2019) Robust anomaly detection in videos using multilevel representations. In: Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):5216–5223

  46. Wang T et al (2019) Generative neural networks for anomaly detection in crowded scenes. IEEE Trans Inf Forensics Secur 14(5):1390–1399

    Article  Google Scholar 

  47. Xu J, Denman S, Sridharan S, Fookes C, Rana R (2011) Dynamic texture reconstruction from sparse codes for unusual event detection in crowded scenes. J-MRE ’11, pp 25–30

  48. Yang J, Yu K, Gong Y, Huang TS (2009) Linear spatial pyramid matching using sparse coding for image classification. IEEE Computer Society, 1794–1801

  49. Yu B, Liu Y, Sun Q (2017) A content-adaptively sparse reconstruction method for abnormal events detection with low-rank property. IEEE Trans Syst Man Cybern Syst 47(4):704–716

    Article  Google Scholar 

  50. Yuan Y, Feng Y, Lu X (2017) Statistical hypothesis detector for abnormal event detection in crowded scenes. IEEE Trans Cybern 47(11):3597–3608

    Article  Google Scholar 

  51. Zhang H, Ngo C (2019) A fine granularity object-level representation for event detection and recounting. IEEE Trans Multim 21(6):1450–1463

    Article  Google Scholar 

  52. Zhang Y, Lu H, Zhang L, Ruan X (2016) Combining motion and appearance cues for anomaly detection. Pattern Recogn 51:443–452

    Article  Google Scholar 

  53. Zhao B, Li F, Xing EP (2011) Online detection of unusual events in videos via dynamic sparse coding. CVPR 2011, Colorado Springs, CO, USA, 2011, pp. 3313–3320. https://doi.org/10.1109/CVPR.2011.5995524

  54. Zhou JT, Du J, Zhu H et al (2019) Anomalynet: An anomaly detection network for video surveillance. IEEE Trans Inf Forensics Secur 14(10):2537–2550

    Article  Google Scholar 

Download references

Acknowledgements

This work was jointly supported by the National Natural Science Foundation of China (61402049), Science and Technology Research Project of the Department of Education of Liaoning Province (LJKZ1019) and Social Science Planning Fund of Liaoning Province (L21BGL002).

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Bohai University, Science and Technology Road, Jinzhou, 121000, Liaoning, China

    Shifeng Li, Yan Cheng, Liuyang Zhao & Yue Wang

Authors
  1. Shifeng Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Yan Cheng
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Liuyang Zhao
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Yue Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Shifeng Li.

Ethics declarations

Competing interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, S., Cheng, Y., Zhao, L. et al. Anomaly detection with multi-scale pyramid grid templates. Multimed Tools Appl 83, 9929–9947 (2024). https://doi.org/10.1007/s11042-023-15569-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15569-6

Keywords