Skip to main content
SpringerLink
Log in

Abnormal crowd behavior detection using high-frequency and spatio-temporal features

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Abnormal crowd behavior detection is an important research issue in computer vision. The traditional methods first extract the local spatio-temporal cuboid from video. Then the cuboid is described by optical flow or gradient features, etc. Unfortunately, because of the complex environmental conditions, such as severe occlusion, over-crowding, etc., the existing algorithms cannot be efficiently applied. In this paper, we derive the high-frequency and spatio-temporal (HFST) features to detect the abnormal crowd behaviors in videos. They are obtained by applying the wavelet transform to the plane in the cuboid which is parallel to the time direction. The high-frequency information characterize the dynamic properties of the cuboid. The HFST features are applied to the both global and local abnormal crowd behavior detection. For the global abnormal crowd behavior detection, Latent Dirichlet allocation is used to model the normal scenes. For the local abnormal crowd behavior detection, Multiple Hidden Markov Models, with an competitive mechanism, is employed to model the normal scenes. The comprehensive experiment results show that the speed of detection has been greatly improved using our approach. Moreover, a good accuracy has been achieved considering the false positive and false negative detection rates.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Reisman, P., Mano, S.A.O., Shashua, A.: Crowd detection in video sequences. In: Proceedings of the IEEE Intelligent Vehicles Symposium, 2004, pp. 66–71, June 2004

  2. Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using social force model. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2009)

  3. Kratz, L., Nishino, K.: Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2009)

  4. Helbing D., Molnár P., Farkas I.J., Bolay K.: Self-organizing pedestrian movement. Environ. Plan. B Plan. Des. 28, 361–383 (2001)

    Article  Google Scholar 

  5. Zhan B., Monekosso D.N., Remagnino P., Velastin S.A., Xu L.-Q.: Crowd analysis: a survey. Mach. Vis. Appl. 19(5–6), 345–357 (2008)

    Article  MATH  Google Scholar 

  6. Haering N., Venetianer P., Lipton A.: The evolution of video surveillance: an overview. Mach. Vis. Appl. 19(5), 279–290 (2008)

    Article  MATH  Google Scholar 

  7. Tu, P., Sebastian, T., Doretto, G., Krahnstoever, N., Rittscher, J., Yu, T.: Unified crowd segmentation. ECCV, pp. 691–704 (2008)

  8. Marques, J.S., Jorge, P.M., Abrantes, A.J., Lemos, J.M.: Tracking Groups of Pedestrians in Video Sequences. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshop 2003, vol. 9, pp. 101 (2003)

  9. Chan, M.T., Hoogs, A., Schmiederer, J., Petersen, M.: Detecting rare events in video using semantic primitives with HMM. In: Proceedings of International Conference on Pattern Recognition, pp. 150–154 (2004)

  10. Gryn, J., Wildes, R., Tsotsos, J.: Detecting motion patterns via direction maps with application to surveillance. In: Proceedings of the IEEE Workshop on Motion and Video Computing, pp. 202–209 (2005)

  11. Johnson, N., Hogg, D.: Learning the distribution of object trajectories for event recognition. In: Proceedings of British Machine Vision Conference, pp. 583–592 (1995)

  12. Chan, M.T., Hoogs, A., Schmiederer, J., Petersen, M.: Detecting rare events in video using semantic primitives with HMM. In: Proceedings of International Conference on Pattern Recognition, pp. 150–154 (2004)

  13. Dee, H., Hogg, D.: Detecting inexplicable behaviour. In: Proceedings of British Macine Vision Conference, pp. 477–486 (2004)

  14. Chan, A.B., Vasconcelos, N.: Mixtures of dynamic textures. In: ICCV’05: Proceedings of the Tenth IEEE International Conference on Computer Vision, vol. 1, pp. 641–647. Washington, DC, USA, IEEE Computer Society (2005)

  15. Andrade, E.L., Blunsden, S., Fisher, R.B.: Modeling crowd scenes for event detection. In: Proceedings of the 18th International Conference on Pattern Recognition (ICPR’06), vol. 01, pp. 175–178. IEEE Computer Society Washington, DC (2006)

  16. Ali, S., Shah, M.: A Lagrangian particle dynamics approach for crowd flow segmentation and stability analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2007, CVPR’07, pp. 1–6, June 2007

  17. Ali, S., Shah, M.: Floor fields for tracking in high density crowd scenes. In: ECCV’08: Proceedings of the 10th European Conference on Computer Vision, pp. 1–14. Springer, Berlin (2008)

  18. Black, M.: Explaining optical flow events with parameterized spatio-temporal models. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 326–332 (1999)

  19. Helbing D., Molnar P.: Social force model for pedestrian dynamics. Phys. Rev. E 51, 4282 (1995)

    Article  Google Scholar 

  20. Blei D., Ng A., Jordan M.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)

    MATH  Google Scholar 

  21. Burrus C.S., Gopinath R.A., Guo H.: Introduction to wavelets and wavelet transforms, pp. 7458. Prentice-Hall, Englewood Cliffs (1998)

    Google Scholar 

  22. Cohen A., Daubechies I., Feauveau J.-C.: Biorthogonal bases of compactly supported wavelets. Commun. Pure Appl. Math. 45(5), 485–560 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  23. Jolliffe I.T.: Principal Component Analysis. Springer, New York (1986)

    Google Scholar 

  24. Fei-Fei, L., Fergus, R., Torralba, A.: Recognizing and learning object categories. Tutorial at ICCV (2005)

  25. Dempster A.P., Laird N.M., Rubin D.R.: Maximum likelihood from incomplete date via the EM algorithm. J. R. Stat. Soc. Ser. B 39(1), 1–38 (1977)

    MathSciNet  MATH  Google Scholar 

  26. Bishop, C.M.: Pattern Recognition and Machine Learning. pp. 76–78. Springer, New York (2006)

  27. Andrade, E., Fisher, R.: Hidden Markov models for optical flow analysis in crowds. In: Proceedings of the 18th International Conference on Pattern Recognition (ICPR06), vol. 01, pp. 460C463. IEEE Computer Society Washington (2006)

  28. Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. In: Proceedings of IEEE, vol. 77, no. 2, pp. 257–285 (1989)

  29. Gui L., Thiran J.P., Paragios N.: Cooperative object segmentation and behavior inference in image sequences. Int. J. Comput. Vis. 84(2), 146–162 (2009)

    Article  Google Scholar 

  30. Unusual crowd activity dataset of University of Minnesota. Available from http://mha.cs.umn.edu/movies/crowdactivity-all.avi

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, 610054, People’s Republic of China

    Bo Wang, Mao Ye, Fengjuan Zhao & Jian Ding

  2. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, People’s Republic of China

    Bo Wang, Mao Ye, Fengjuan Zhao & Jian Ding

  3. School of Information Technology and Electronic Engineering, The University of Queensland, Brisbane, QLD, 4072, Australia

    Xue Li

Authors
  1. Bo Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mao Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xue Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fengjuan Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jian Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bo Wang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wang, B., Ye, M., Li, X. et al. Abnormal crowd behavior detection using high-frequency and spatio-temporal features. Machine Vision and Applications 23, 501–511 (2012). https://doi.org/10.1007/s00138-011-0341-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-011-0341-0

Keywords

Search

Navigation