Skip to main content
SpringerLink
Log in

Real-time abnormal situation detection based on particle advection in crowded scenes

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

This paper presents a real-time abnormal situation detection method in crowded scenes based on the crowd motion characteristics including the particle energy and the motion directions. The particle energy is determined by computation of optical flow derived from two adjacent frames. The particle energy is modified by multiplying the foreground to background ratio. The motion directions are measured by mutual information of the direction histograms of two neighboring motion vector fields. Mutual information is used to measure the similarity between two direction histograms derived from three adjacent frames. The direction probability distribution for each frame can be directly estimated from the direction histogram by dividing the entries by the total number of the vectors. A metric for all the video frames is computed using normalized mutual information to detect the abnormal situation. Both the modified particle energy and mutual information of direction histograms contribute to the detection of the abnormal events. Furthermore, the dynamic abnormality is measured to detect the dynamical movement associated with severe change in the motion state according to the spatio-temporal characteristics. In experiments, we will show that the proposed method detects the abnormal situations effectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Ali, S., Shah, M.: A lagrangian particle dynamics approach for crowd flow segmentation and stability analysis. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1–6 (2007). doi:10.1109/CVPR.2007.382977

  2. Andrade, E.L., Blunsden, S., Fisher, R.B.: Hidden markov models for optical flow analysis in crowds. In: ICPR ’06, Proceedings of the 18th International Conference on Pattern Recognition, vol. 01, pp. 460–463. IEEE Computer Society, Washington, DC (2006). doi:10.1109/ICPR.2006.621

  3. Baker, S., Matthews, I.: Lucas-kanade 20 years on: a unifying framework. Int. J. Comput. Vis. 56(3), 221–255 (2004)

    Article  Google Scholar 

  4. Barron, J., Fleet, D., Beauchemin, S., Burkitt, T.A.: Performance of optical flow techniques. In: Proceedings CVPR ’92, IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 236–242 (1992). doi:10.1109/CVPR.1992.223269

  5. Black, M.J, Anandan, P.: A framework for the robust estimation of optical flow. In: Proceedings of Fourth International Conference on Computer Vision, pp. 231–236. IEEE press, New York (1993)

  6. Bouguet, J.Y.: Pyramidal implementation of the affine lucas kanade feature tracker description of the algorithm. Intel Corp. 5 (2001)

  7. Bruhn, A., Weickert, J., Feddern, C., Kohlberger, T., Schnorr, C.: Variational optical flow computation in real time. IEEE Trans. Image Process. 14(5), 608–615 (2005a)

    Article  MathSciNet  Google Scholar 

  8. Bruhn, A., Weickert, J., Kohlberger, T., Schnörr, C.: Discontinuity-preserving computation of variational optic flow in real-time. In: Scale Space and PDE Methods in Computer Vision, pp. 279–290. Springer, Berlin (2005b)

  9. Duong, T., Bui, H., Phung, D., Venkatesh, S.: Activity recognition and abnormality detection with the switching hidden semi-markov model. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR, vol. 1, pp. 838–845 (2005). doi:10.1109/CVPR.2005.61

  10. Hu, M., Ali, S., Shah, M.: Detecting global motion patterns in complex videos. In: ICPR 2008, 19th International Conference on Pattern Recognition, pp. 1–5 (2008). doi:10.1109/ICPR.2008.4760950

  11. Ihaddadene, N., Djeraba, C.: Real-time crowd motion analysis. In: 19th International Conference on Pattern Recognition (ICPR 2008), pp. 1–4, IEEE, Tampa (2008). doi:10.1109/ICPR.2008.4761041

  12. Jaynes, E.T.: Information theory and statistical mechanics. Phys. Rev. 106(4), 620 (1957)

    Article  MATH  MathSciNet  Google Scholar 

  13. Li, J., Gong, S., Xiang, T.: Global behaviour inference using probabilistic latent semantic analysis. In: BMVC’08, p. 10

  14. Li, W., Wu, X., Zhao, H.A.: New techniques of foreground detection, segmentation and density estimation for crowded objects motion analysis. JIP 19, 190–200 (2011)

    Google Scholar 

  15. Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence, vol. 2, pp. 674–679. Morgan Kaufmann Publishers Inc., San Francisco (1981). http://dl.acm.org/citation.cfm?id=1623264.1623280

  16. Ma, R., Li, L., Huang, W., Tian, Q.: On pixel count based crowd density estimation for visual surveillance. In: IEEE Conference on Cybernetics and Intelligent Systems, vol. 1, pp. 170–173 (2004). doi:10.1109/ICCIS.2004.1460406

  17. Marana, A., Velastin, S., Costa, L., Lotufo, R.: Estimation of crowd density using image processing. In: IEE Colloquium on Image Processing for Security Applications (Digest No.: 1997/074), pp. 11/1–11/8 (1997). doi:10.1049/ic:19970387

  18. Marana, A.N., Cavenaghi, M.A., Ulson, R.S., Drumond, F.L.: Real-time crowd density estimation using images. Adv. Vis. Comput. 3804, 355–362 (2005)

    Article  Google Scholar 

  19. Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using social force model. In: CVPR 2009, IEEE Conference on Computer Vision and Pattern Recognition, pp. 935–942 (2009). doi:10.1109/CVPR.2009.5206641

  20. Nam, Y.: Crowd flux analysis and abnormal event detection in unstructured and structured scenes. Multimed. Tools Appl. pp. 1–29 (2013a)

  21. Nam, Y.: Dr. yunyoung nam-google sites (2013b). https://sites.google.com/site/yynams/crowd-activity

  22. Nam, Y., Rho, S., Park, J.: Intelligent video surveillance system: 3-tier context-aware surveillance system with metadata. Multimed. Tools Appl. 57, 315–334 (2012). doi:10.1007/s11042-010-0677-x

  23. Nam, Y., Rho, S., Park, J.: Inference topology of distributed camera networks with multiple cameras. Multimed. Tools Appl. 67(1), 289–309 (2013). doi:10.1007/s11042-012-0997-0

  24. Papoulis, A., Pillai, S.U.: Probability, random variables, and stochastic processes. Tata McGraw-Hill Education, New Delhi (2002)

  25. Pavón, J., Gómez-Sanz, J., Fernández-Caballero, A., Valencia-Jiménez, J.J.: Development of intelligent multisensor surveillance systems with agents. Robot Auton. Syst. 55(12), 892–903 (2007). doi:10.1016/j.robot.2007.07.009

  26. Pluim, J., Maintz, J., Viergever, M.: Mutual-information-based registration of medical images: a survey. IEEE Trans. Med. Imaging 22(8), 986–1004 (2003). doi:10.1109/TMI.2003.815867

    Article  Google Scholar 

  27. Rahmalan, H., Nixon, M., Carter, J.: On crowd density estimation for surveillance. In: The Institution of Engineering and Technology Conference on Crime and Security, pp. 540–545 (2006)

  28. UMN: Umn: Unusual crowd activity dataset of university of minnesota (2013). http://mha.cs.umn.edu/Movies/Crowd-Activity-All.avi

  29. Valera, M., Velastin, S.: Intelligent distributed surveillance systems: a review. In: IEE Proceedings on Vision, Image and Signal Processing, vol. 152(2), pp. 192–204 (2005). doi:10.1049/ip-vis:20041147

  30. Van-Trees, H.: Detection, Estimation, and Modulation Theory. Part I. Wiley, New York (2001)

  31. Wang, X., Ma, X., Grimson, W.E.L.: Unsupervised activity perception in crowded and complicated scenes using hierarchical bayesian models. IEEE Trans. Pattern Anal. Mach. Intell. 31, 539–555 (2009). doi:10.1109/TPAMI.2008.87, http://dl.acm.org/citation.cfm?id=1512152.1512378

  32. Wu, S., Moore, B., Shah, M.: Chaotic invariants of lagrangian particle trajectories for anomaly detection in crowded scenes. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2054–2060 (2010). doi:10.1109/CVPR.2010.5539882

  33. Yang, H., Nam, Y., Cho, W.D., Choi, Y.J.: Adaptive background modeling for effective ghost removal and robust left object detection. In: 2nd International Conference on Information Technology Convergence and Services (ITCS), pp. 1–6 (2010). doi:10.1109/ITCS.2010.5581283

  34. Zhang, D., Gatica-Perez, D., Bengio, S., McCowan, I.: Semi-supervised adapted hmms for unusual event detection. In: CVPR 2005. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 611–618 (2005). doi:10.1109/CVPR.2005.316

  35. Zhong, H., Shi, J., Visontai, M.: Detecting unusual activity in video. In: CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. II-819–II-826 (2004). doi:10.1109/CVPR.2004.1315249

  36. Zivkovic, Z.: Improved adaptive Gaussian mixture model for background subtraction. In: 17th International Conference on (ICPR’04) Proceedings of the Pattern Recognition, vol. 2, pp. 28–31. IEEE Computer Society, Washington, DC (2004). doi:10.1109/ICPR.2004.479

Download references

Acknowledgments

They would like to thank Dianne Greco for previewing our manuscript.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Soonchunhyang University, Asan, 336-745, Korea

    Yunyoung Nam

  2. Department of Electrical and Computer Engineering, Stony Brook University-SUNY, Stony Brook, NY, USA

    Sangjin Hong

Authors
  1. Yunyoung Nam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sangjin Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yunyoung Nam.

Additional information

This research was supported by the Soonchunhyang University Research Fund and also supported by the International Collaborative R&D Program of the Ministry of Knowledge Economy (MKE), the Korean government, as a result of Development of Security Threat Control System with Multi-Sensor Integration and Image Analysis Project, 2010-TD-300802-002.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nam, Y., Hong, S. Real-time abnormal situation detection based on particle advection in crowded scenes. J Real-Time Image Proc 10, 771–784 (2015). https://doi.org/10.1007/s11554-014-0424-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-014-0424-z

Keywords

Search

Navigation