Skip to main content
SpringerLink
Log in

Detection and localization of crowd behavior using a novel tracklet-based model

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

In this paper, two novel descriptors are introduced to detect and localize abnormal behaviors in crowded scenes. The first proposed descriptor is based on the orientation and magnitude of short trajectories extracted by tracking interest points in spatio-temporal 3D patches. The proposed descriptor employs a novel simplified Histogram of Oriented Tracklets (sHOT), which is shown to be very effective in the task of crowd abnormal behavior detection. In this scheme, abnormal behaviors are detected at different levels, namely spatio-temporal level and frame level. By combining the first proposed descriptor and the dense optical flow model, we propose our second framework which is able to localize the abnormal behavior areas in video sequences. The evaluation of our simple but yet effective descriptors on different state-of-the-art datasets, namely UCSD, UMN and Violence in Crowds yields very promising results in abnormality detection and outperforming different former state-of-the-art descriptors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. http://www.ces.clemson.edu/stb/klt/.

References

  1. Chen K, Gong S, Xiang T, Change Loy C (2013) Cumulative attribute space for age and crowd density estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2467–2474

  2. Wu S, San Wong H (2012) Crowd motion partitioning in a scattered motion field. IEEE Trans Syst Man Cybern Part B Cybern 42:1443–1454

    Article  Google Scholar 

  3. Bera A, Manocha D (2014) Realtime multilevel crowd tracking using reciprocal velocity obstacles. arXiv preprint arXiv:1402.2826

  4. Fang Z, Fei F, Fang Y, Lee C, Xiong N, Shu L et al (2016) Abnormal event detection in crowded scenes based on deep learning. Multimed Tools Appl, 1–23

  5. Hu X, Hu S, Huang Y, Zhang H, Wu H (2016) Video anomaly detection using deep incremental slow feature analysis network. IET Comput Vis 10:265

    Article  Google Scholar 

  6. Mousavi H, Nabi M, Galoogahi HK, Perina A, Murino V (2015) Abnormality detection with improved histogram of oriented tracklets. In: International Conference on Image Analysis and Processing, pp 722–732

    Chapter  Google Scholar 

  7. Mousavi H, Nabi M, Kiani H, Perina A, Murino V (2015) Crowd motion monitoring using tracklet-based commotion measure. In: Image Processing (ICIP), 2015 IEEE International Conference, pp 2354–2358

  8. Ravanbakhsh M, Nabi M, Mousavi H, Sangineto E, Sebe N (2016) Plug-and-play CNN for crowd motion analysis: an application in abnormal event detection. arXiv preprint arXiv:1610.00307

  9. Sabokrou M, Fathy M, Hoseini M (2016) Video anomaly detection and localisation based on the sparsity and reconstruction error of auto-encoder. Electron Lett 52:1122–1124

    Article  Google Scholar 

  10. Sabokrou M, Fathy M, Hoseini M, Klette R (2015) Real-time anomaly detection and localization in crowded scenes. Proceedings of the IEEE Conference on Computer Vision Pattern Recognition Workshops, pp 56–62

  11. Wang B, Ye M, Li X, Zhao F, Ding J (2012) Abnormal crowd behavior detection using high-frequency and spatio-temporal features. Mach Vis Appl 23:501–511

    Article  Google Scholar 

  12. Wu S, Wong H-S, Yu Z (2014) A Bayesian model for crowd escape behavior detection. IEEE Trans Circuits Syst Video Technol 24:85–98

    Article  Google Scholar 

  13. Zhou S, Shen W, Zeng D, Fang M, Wei Y, Zhang Z (2016) Spatial–temporal convolutional neural networks for anomaly detection and localization in crowded scenes. Signal Process Image Commun 47:358–368

    Article  Google Scholar 

  14. Wijermans A, Jorna R, Jager E, Van Vliet T (2007) Modelling crowd dynamics influence factors related to the probability of a riot

  15. Rabaud V, Belongie S (2006) Counting crowded moving objects. 2006 IEEE Computer Society Conference on Computer Vision Pattern Recognition (CVPR’06), pp 705–711

  16. Rittscher J, Tu PH, Krahnstoever N (2005) Simultaneous estimation of segmentation and shape. 2005 IEEE Computer Society Conference on Computer Vision Pattern Recognition (CVPR’05), pp 486–493

  17. Kratz L, Nishino K (2009) Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models. Computer Vision Pattern Recognition CVPR 2009 IEEE Conference, pp 1446–1453

  18. Kratz L, Nishino K (2010) Tracking with local spatio-temporal motion patterns in extremely crowded scenes. Computer Vision Pattern Recognition (CVPR) 2010 IEEE Conference, pp 693–700

  19. Krausz B, Bauckhage C (2011) Analyzing pedestrian behavior in crowds for automatic detection of congestions. Computer vision workshops (ICCV workshops) 2011 IEEE international conference, pp 144–149

  20. Mehran R, Oyama A, Shah M (2009) Abnormal crowd behavior detection using social force model. Computer Vision Pattern Recognition CVPR 2009 IEEE Conference, pp 935–942

  21. Wang H, Kläser A, Schmid C, Liu C-L (2011) Action recognition by dense trajectories. Computer Vision Pattern Recognition (CVPR) 2011 IEEE Conference, pp 3169–3176

  22. Raptis M, Soatto S (2010) Tracklet descriptors for action modeling and video analysis. In: European conference on computer vision, pp 577–590

    Chapter  Google Scholar 

  23. Lucas BD, Kanade T (1981) An iterative image registration technique with an application to stereo vision. IJCAI, 674–679

  24. Alvarez L, Weickert J, Sánchez J (2000) Reliable estimation of dense optical flow fields with large displacements. Int J Comput Vis 39:41–56

    Article  Google Scholar 

  25. Mahadevan V, Li W, Bhalodia V, Vasconcelos N (2010) Anomaly detection in crowded scenes. CVPR, p 250

  26. Hassner T, Itcher Y, Kliper-Gross O (2012) Violent flows: Real-time detection of violent crowd behavior. 2012 IEEE Computer Society Conference on Computer Vision Pattern Recognition Workshops, pp 1–6

  27. Wang T, Snoussi H (2012) Histograms of optical flow orientation for visual abnormal events detection. Advanced video signal-based surveillance (AVSS) 2012 IEEE Ninth International Conference, pp 13–18

  28. Zhao T, Nevatia R (2003) Bayesian human segmentation in crowded situations, vol 2. Computer Vision Pattern Recognition 2003 Proceedings 2003 IEEE Computer Society Conference, pp 459–466

  29. Shi J, Tomasi C (1994) Good features to track. Computer Vision Pattern Recognition 1994 Proceedings CVPR’94 1994 IEEE Computer Society Conference, pp 593–600

  30. Krausz B, Bauckhage C (2012) Loveparade 2010: Automatic video analysis of a crowd disaster. Comput Vis Image Underst 116:307–319

    Article  Google Scholar 

  31. Helbing D, Molnar P (1995) Social force model for pedestrian dynamics. Phys Rev E 51:4282

    Article  Google Scholar 

  32. Solmaz B, Moore BE, Shah M (2012) Identifying behaviors in crowd scenes using stability analysis for dynamical systems. IEEE Trans Pattern Anal Mach Intell 34:2064–2070

    Article  Google Scholar 

  33. Hu M, Ali S, Shah M (2008) Detecting global motion patterns in complex videos. Pattern Recognition ICPR 2008 19th International Conference, pp 1–5

  34. Duda RO, Hart PE, Stork DG (2012) Pattern classification. Wiley, Hoboken

    MATH  Google Scholar 

  35. Blei DM, Ng AY, Jordan MI (2003) Latent Dirichlet allocation. J Mach Learn Res 3:993–1022

    MATH  Google Scholar 

  36. Li W, Mahadevan V, Vasconcelos N (2014) Anomaly detection and localization in crowded scenes. IEEE Trans Pattern Anal Mach Intell 36:18–32

    Article  Google Scholar 

  37. Rabiee H, Haddadnia J, Mousavi H, Nabi M, Murino V, Sebe N (2016) Emotion-based crowd representation for abnormality detection. arXiv preprint: arXiv:1607.07646

  38. Rabiee H, Haddadnia J, Mousavi H (2016) Crowd behavior representation: an attribute-based approach. SpringerPlus 5:1179

    Article  Google Scholar 

  39. Mousavi H, Mohammadi S, Perina A, Chellali R, Murino V (2015) Analyzing tracklets for the detection of abnormal crowd behavior. 2015 IEEE Winter Conference on Applications of Computer Vision, pp 148–155

  40. Ravanbakhsh M, Mousavi H, Rastegari M, Murino V, Davis LS (2015) Action recognition with image based CNN features. arXiv preprint arXiv:1512.03980

  41. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60:91–110

    Article  Google Scholar 

  42. Rabiee H, Haddadnia J, Mousavi H, Kalantarzadeh M, Nabi M, Murino V (2016) Novel dataset for fine-grained abnormal behavior understanding in crowd. Advanced video signal based surveillance (AVSS) 13th IEEE International Conference, pp 95–101

  43. Adam A, Rivlin E, Shimshoni I, Reinitz D (2008) Robust real-time unusual event detection using multiple fixed-location monitors. IEEE Trans Pattern Anal Mach Intell 30:555–560

    Article  Google Scholar 

  44. Cong Y, Yuan J, Liu J (2011) Sparse reconstruction cost for abnormal event detection. Computer Vision Pattern Recognition (CVPR) 2011 IEEE Conference, pp 3449–3456

  45. Laptev I, Marszalek M, Schmid C, Rozenfeld B (2008) Learning realistic human actions from movies. Computer Vision Pattern Recognition CVPR 2008 IEEE Conference, pp 1–8

  46. Yeffet L, Wolf L (2009) Local trinary patterns for human action recognition. 2009 IEEE 12th International Conference on Computer Vision, pp 492–497

  47. Mousavi H, Galoogahi HK, Perina A, Murino V (2016) Detecting abnormal behavioral patterns in crowd scenarios. In: Toward robotic socially believable behaving systems-volume II., Springer, pp 185–205

  48. Xu D, Ricci E, Yan Y, Song J, Sebe N (2015) Learning deep representations of appearance and motion for anomalous event detection. arXiv preprint arXiv:1510.01553

  49. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. 2005 IEEE Computer Society Conference on Computer Vision Pattern Recognition (CVPR’05), pp 886–893

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Faculty of Mechatronics, Karaj Branch, Islamic Azad University, Karaj, Iran

    Hamidreza Rabiee

  2. Polytechnique Montréal, Montréal, QC, H3C 3A7, Canada

    Hossein Mousavi

  3. DISI, University of Trento, Trento, Italy

    Moin Nabi

  4. DITEN, University of Genoa, Genova, Italy

    Mahdyar Ravanbakhsh

Authors
  1. Hamidreza Rabiee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hossein Mousavi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Moin Nabi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mahdyar Ravanbakhsh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hamidreza Rabiee.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rabiee, H., Mousavi, H., Nabi, M. et al. Detection and localization of crowd behavior using a novel tracklet-based model. Int. J. Mach. Learn. & Cyber. 9, 1999–2010 (2018). https://doi.org/10.1007/s13042-017-0682-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-017-0682-8

Keywords

Search

Navigation