Skip to main content
SpringerLink
Log in

A thermodynamics-inspired feature for anomaly detection on crowd motions in surveillance videos

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Identification of abnormal behaviors in surveillance videos of crowds plays an important role in public security monitoring. However, detecting abnormal crowd behaviors is challenging in that movements of individuals are usually random and unpredictable, and the occlusions caused by over-crowding make the task more difficult. In this paper, we introduce thermodynamic micro-statistics theory to detect and localize abnormal behaviors in crowded scenes based on Boltzmann Entropy. For this purpose, the scene of interest is modeled as moving particles turned out from a general optical flow algorithm. The particles are grouped into a set of prototypes according to their speeds and directions of moving, and a histogram is established to figure out how the particles distribute over the prototypes. Here, Boltzmann Entropy is computed from the histogram for each video clip to characterize the chaos degree of crowd motion. By means of such feature extraction, the crowd motion patterns can be represented as a time series. We find that when most people behave anomaly in an area under surveillance, the corresponding entropy value will increase remarkably in comparison with those of normal cases. This motives us to make use of Boltzmann Entropy to distinguish the collective behaviors of people under emergent circumstances from their normal behaviors by evaluating how significantly the current feature value fits into the Gaussian model of normal cases. We validate our method extensively for anomaly detection and localization. The experimental results show promising performance compared with the state of the art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. http://pan.baidu.com/s/1qWwTY9m

References

  1. Andrade EL, Blunsden S, Fisher RB Modelling Crowd Scenes for Event Detection. In: Pattern Recognition, 2006. ICPR 2006. 18th International Conference on, 0–0 0 2006. pp 175–178. doi:10.1109/ICPR.2006.806

  2. Basharat A, Gritai A, Shah M Learning object motion patterns for anomaly detection and improved object detection. In: Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on, 23–28 June 2008 2008. pp 1–8. doi:10.1109/CVPR.2008.4587510

  3. Brox T, Bruhn A, Papenberg N, Weickert J (2004) High accuracy optical flow estimation based on a theory for warping. In: Pajdla T, Matas J (eds) Computer vision - ECCV 2004, Lecture Notes in Computer Science, vol 3024. Springer, Berlin Heidelberg, pp. 25–36. doi:10.1007/978-3-540-24673-2_3

    Chapter  Google Scholar 

  4. Cong Y, Yuan J, Liu J (2013) Abnormal event detection in crowded scenes using sparse representation. Pattern Recogn 46(7):1851–1864. doi:10.1016/j.patcog.2012.11.021

    Article  Google Scholar 

  5. Dalal N, Triggs B Histograms of oriented gradients for human detection. In: Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, 25–25 June 2005 2005. pp 886–893 vol. 881. doi:10.1109/CVPR.2005.177

  6. Gu X, Cui J, Zhu Q (2014) Abnormal crowd behavior detection by using the particle entropy. Optik – International Journal for Light and Electron Optics 125(14):3428–3433. doi:10.1016/j.ijleo.2014.01.041

    Article  Google Scholar 

  7. Haering N, Venetianer P, Lipton A (2008) The evolution of video surveillance: an overview. Mach Vis Appl 19(5–6):279–290. doi:10.1007/s00138-008-0152-0

    Article  Google Scholar 

  8. Halliday D, Resnick R, Walker J (2010) Fundamentals of physics extended 9 edition. Wiley:550-560

  9. Helbing D, Molnár P (1995) Social force model for pedestrian dynamics. Phys Rev E 51(5):4282–4286

    Article  Google Scholar 

  10. Helbing D, Johansson A, Al-Abideen HZ (2007) Dynamics of crowd disasters: an empirical study. Phys Rev E 75(4). doi:10.1103/PhysRevE.75.046109

  11. Jie F, Chao Z, Pengwei H Online anomaly detection in videos by clustering dynamic exemplars. In: Image Processing (ICIP), 2012 19th IEEE International Conference on, Sept. 30 2012-Oct. 3 2012 2012. pp 3097–3100. doi:10.1109/ICIP.2012.6467555

  12. Kaltsa V, Briassouli A, Kompatsiaris I, Hadjileontiadis LJ, Strintzis MG (2015) Swarm intelligence for detecting interesting events in crowded environments. Image Processing, IEEE Transactions on 24(7):2153–2166. doi:10.1109/TIP.2015.2409559

    Article  MathSciNet  Google Scholar 

  13. Mahadevan V, Weixin L, Bhalodia V, Vasconcelos N Anomaly detection in crowded scenes. In: Computer Vision and Pattern Recognition (CVPR), 2010 I.E. Conference on, 13–18 June 2010 2010. pp 1975–1981. doi:10.1109/CVPR.2010.5539872

  14. Mehran R, Oyama A, Shah M Abnormal crowd behavior detection using social force model. In: Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, 20–25 June 2009 2009. pp 935–942. doi:10.1109/CVPR.2009.5206641

  15. Nam Y (2014) Crowd flux analysis and abnormal event detection in unstructured and structured scenes. Multimed Tools Appl 72(3):3001–3029. doi:10.1007/s11042-013-1593-7

    Article  Google Scholar 

  16. Pathan S, Al-Hamadi A, Michaelis B (2011) Using conditional random field for crowd behavior analysis. In: Koch R, Huang F (eds) Computer vision – ACCV 2010 workshops, Lecture Notes in Computer Science, vol 6468. Springer, Berlin Heidelberg, pp. 370–379. doi:10.1007/978-3-642-22822-3_37

    Chapter  Google Scholar 

  17. Raghavendra R, Del Bue A, Cristani M, Murino V Optimizing interaction force for global anomaly detection in crowded scenes. In: Computer Vision Workshops (ICCV Workshops), 2011 I.E. International Conference on, 6–13 Nov. 2011 2011. pp 136–143. doi:10.1109/ICCVW.2011.6130235

  18. Reddy V, Sanderson C, Lovell BC Improved anomaly detection in crowded scenes via cell-based analysis of foreground speed, size and texture. In: Computer Vision and Pattern Recognition Workshops (CVPRW), 2011 I.E. Computer Society Conference on, 20–25 June 2011 2011. pp 55–61. doi:10.1109/CVPRW.2011.5981799

  19. Sethi RJ, Roy-Chowdhury AK (2010) Modeling and recognition of complex multi-person interactions in video. Paper presented at the Proceedings of the 1st ACM international workshop on Multimodal pervasive video analysis, Firenze, Italy,

  20. Shandong W, Moore BE, Shah M Chaotic invariants of Lagrangian particle trajectories for anomaly detection in crowded scenes. In: Computer Vision and Pattern Recognition (CVPR), 2010 I.E. Conference on, 13–18 June 2010 2010. pp 2054–2060. doi:10.1109/CVPR.2010.5539882

  21. Shannon CE (1948) A mathematical theory of communication. Bell System Technical Journal 27:379–423

    Article  MathSciNet  MATH  Google Scholar 

  22. Si W, Hau-San W, Zhiwen Y (2014) A Bayesian model for crowd escape behavior detection. Circuits and Systems for Video Technology, IEEE Transactions on 24(1):85–98. doi:10.1109/TCSVT.2013.2276151

    Article  Google Scholar 

  23. Susan S, Hanmandlu M (2013) Unsupervised detection of nonlinearity in motion using weighted average of non-extensive entropies. SIViP:1–15. doi:10.1007/s11760-013-0464-z

  24. Tian C, Xinyu W, Jinnian G, Shiqi Y, Yangsheng X Abnormal crowd motion analysis. In: Robotics and Biomimetics (ROBIO), 2009 I.E. International Conference on, 19–23 Dec. 2009 2009. pp 1709–1714. doi:10.1109/ROBIO.2009.5420408

  25. Tu P, Sebastian T, Doretto G, Krahnstoever N, Rittscher J, Yu T (2008) Unified crowd segmentation. In: Forsyth D, Torr P, Zisserman A (eds) Computer vision – ECCV 2008, Lecture Notes in Computer Science, vol 5305. Springer, Berlin Heidelberg, pp. 691–704. doi:10.1007/978-3-540-88693-8_51

    Chapter  Google Scholar 

  26. UMN Unusual crowd activity dataset of University of Minnesota. available from http://mhacsumnedu/movies/crowd-activity-allavi

  27. Wang X, Tieu K, Grimson E (2006) Learning semantic scene models by trajectory analysis. In: Leonardis A, Bischof H, Pinz A (eds) Computer vision – ECCV 2006, Lecture Notes in Computer Science, vol 3953. Springer, Berlin Heidelberg, pp. 110–123. doi:10.1007/11744078_9

    Chapter  Google Scholar 

  28. Weixin L, Mahadevan V, Vasconcelos N (2014) Anomaly detection and localization in crowded scenes. Pattern Analysis and Machine Intelligence, IEEE Transactions on 36(1):18–32. doi:10.1109/TPAMI.2013.111

    Article  Google Scholar 

  29. Xinyi C, Qingshan L, Mingchen G, Metaxas DN Abnormal detection using interaction energy potentials. In: Computer Vision and Pattern Recognition (CVPR), 2011 I.E. Conference on, 20–25 June 2011 2011. pp 3161–3167. doi:10.1109/CVPR.2011.5995558

  30. Xiong G, Cheng J, Wu X, Chen Y-L, Ou Y, Xu Y (2012) An energy model approach to people counting for abnormal crowd behavior detection. Neurocomputing 83(0):121–135. doi:10.1016/j.neucom.2011.12.007

    Article  Google Scholar 

  31. Yang S Gene Ranking Using R-Metric. In: Computational Intelligence and Security, 2006 International Conference on, Nov. 2006 2006. pp 465–469. doi:10.1109/ICCIAS.2006.294178

  32. Yang C, Junsong Y, Ji L Sparse reconstruction cost for abnormal event detection. In: Computer Vision and Pattern Recognition (CVPR), 2011 I.E. Conference on, 20–25 June 2011 2011. pp 3449–3456. doi:10.1109/CVPR.2011.5995434

  33. Zhang X, Yang S, Tang YY, Zhang W (2015) Crowd Motion Monitoring with Thermodynamics-Inspired Feature. North America, mar. 2015., AAAI Conference on Artificial Intelligence

Download references

Acknowledgments

This work is supported by NSFC under grant No. 61472087.

Author information

Authors and Affiliations

  1. Shanghai Key Laboratory of Intelligent Information Processing, College of Computer Science, Fudan University, Shanghai, 201203, China

    Xinfeng Zhang & Su Yang

  2. Department of Computer and Information Science, University of Macau, Macau, China

    Yuan Yan Tang

  3. Department of Software Engineering, China University of Petroleum, Qingdao, 266580, China

    Weishan Zhang

Authors
  1. Xinfeng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Su Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuan Yan Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Weishan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Su Yang.

Electronic Supplementary Material

ESM 1

(PDF 645 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, X., Yang, S., Tang, Y.Y. et al. A thermodynamics-inspired feature for anomaly detection on crowd motions in surveillance videos. Multimed Tools Appl 75, 8799–8826 (2016). https://doi.org/10.1007/s11042-015-3101-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-3101-8

Keywords

Search

Navigation