Abstract
Detecting and localizing anomalies in crowded scenes is an ongoing challenge for public security. Existing approaches are mainly based on patches and trajectories. However, they fall short in semantic understanding of scenes and tackling the depth-of-field problem, respectively. In this paper, we put forward a novel object and patch based framework for anomaly detection and localization. Specifically, we propose to colorize images for precise object detection in dim scenarios. Categories of the objects are used for appearance anomaly justification. For motion anomaly, we propose a new patch-based algorithm which is robust to the depth-of-field problem, which can also be used to detect location anomalies. Besides, a new object re-targeting method is proposed to find the missing objects in detection. It can also handle drift and occlusions in tracking, which can avoid false alarms. Extensive experiments are conducted on several benchmark datasets for anomaly detection. The results show that the proposed method can achieve comparable accuracy in anomaly detection with state-of-the-arts methods and at the same time, localize anomalies precisely.
Similar content being viewed by others
References
Antonakaki P, Kosmopoulos D, Perantonis SJ (2009) Detecting abnormal human behaviour using multiple cameras. Signal Process 89(9):1723–1738
Calderara S, Heinemann U, Prati A, Cucchiara R, Tishby N (2011) Detecting anomalies in people’s trajectories using spectral graph analysis. Comput Vis Image Underst 115(8):1099–1111
Cong Y, Yuan J, Liu J (2013) Abnormal event detection in crowded scenes using sparse representation. Pattern Recogn 46(7):1851–1864
Coşar S, Donatiello G, Bogorny V, Garate C, Alvares LO, Brémond F (2017) Toward abnormal trajectory and event detection in video surveillance. IEEE Trans Circuits Syst Video Technol 27(3):683–695
Dai J, Li Y, He K, Sun J (2016) R-fcn: object detection via region-based fully convolutional networks. In: Advances in neural information processing systems, pp 379–387
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005. CVPR 2005. IEEE computer society conference on computer vision and pattern recognition, vol 1. IEEE, pp 886– 893
Dalal N, Triggs B, Schmid C (2006) Human detection using oriented histograms of flow and appearance. In: European conference on computer vision, Springer, pp 428–441
Feng Y, Yuan Y, Lu X (2017) Learning deep event models for crowd anomaly detection. Neurocomputing 219:548–556
Hasan M, Choi J, Neumann J, Roy-Chowdhury AK, Davis LS (2016) Learning temporal regularity in video sequences. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 733–742
He Z, Li X, You X, Tao D, Tang YY (2016) Connected component model for multi-object tracking. IEEE Trans Image Process 25(8):3698–3711
Henriques JF, Caseiro R, Martins P, Batista J (2015) High-speed tracking with kernelized correlation filters. IEEE Trans Pattern Anal Mach Intell 37(3):583–596
Hinami R, Mei T, Satoh S (2017) Joint detection and recounting of abnormal events by learning deep generic knowledge. In: ICCV, pp 3639–3647
Hu W, Xiao X, Fu Z, Xie D, Tan T, Maybank S (2006) A system for learning statistical motion patterns. IEEE Trans Pattern Anal Mach Intell 28(9):1450–1464
Jiang F, Yuan J, Tsaftaris SA, Katsaggelos AK (2011) Anomalous video event detection using spatiotemporal context. Comput Vis Image Underst 115 (3):323–333
Khalid S (2010) Activity classification and anomaly detection using m-mediods based modelling of motion patterns. Pattern Recogn 43(10):3636–3647
Kim J, Grauman K (2009) Observe locally, infer globally: a space-time mrf for detecting abnormal activities with incremental updates. In: 2009. CVPR 2009. IEEE conference on computer vision and pattern recognition, IEEE, pp 2921–2928
Lee S, Kim HG, Ro YM (2018) Stan: spatio-temporal adversarial networks for abnormal event detection. arXiv:1804.08381
Li Q, Li W (2016) A novel framework for anomaly detection in video surveillance using multi-feature extraction. In: 2016 9th international symposium on computational intelligence and design (ISCID), vol 1. IEEE, pp 455–459
Liu Q, Liu B, Yu N (2017) Tccf: tracking based on convolutional neural network and correlation filters. In: International conference on image and graphics, Springer, pp 316–327
Liu W, Luo W, Lian D, Gao S (2017) Future frame prediction for anomaly detection–a new baseline. arXiv:1712.09867
Lu C, Shi J, Jia J (2013) Abnormal event detection at 150 fps in matlab. In: Proceedings of the IEEE international conference on computer vision, pp 2720–2727
Luo W, Liu W, Gao S (2017) A revisit of sparse coding based anomaly detection in stacked rnn framework. ICCV 1(2):3
Mahadevan V, Li W, Bhalodia V, Vasconcelos N (2010) Anomaly detection in crowded scenes. In: 2010 IEEE conference on computer vision and pattern recognition (CVPR), IEEE, pp 1975–1981
Mehran R, Oyama A, Shah M (2009) Abnormal crowd behavior detection using social force model. In: 2009. CVPR 2009. IEEE conference on computer vision and pattern recognition, IEEE, pp 935–942
Morris BT, Trivedi MM (2011) Trajectory learning for activity understanding: unsupervised, multilevel, and long-term adaptive approach. IEEE Trans Pattern Anal Mach Intell 33(11):2287–2301
Piciarelli C, Foresti GL (2006) On-line trajectory clustering for anomalous events detection. Pattern Recogn Lett 27(15):1835–1842
Piciarelli C, Micheloni C, Foresti GL (2008) Trajectory-based anomalous event detection. IEEE Trans Circuits Syst Video Technol 18(11):1544–1554
Ravanbakhsh M, Nabi M, Sangineto E, Marcenaro L, Regazzoni C, Sebe N (2017) Abnormal event detection in videos using generative adversarial nets. In: 2017 IEEE international conference on image processing (ICIP), IEEE, pp 1577–1581
Sabokrou M, Fathy M, Hoseini M, Klette R (2015) Real-time anomaly detection and localization in crowded scenes. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 56–62
Sabokrou M, Fayyaz M, Fathy M, Klette R (2017) Deep-cascade: cascading 3d deep neural networks for fast anomaly detection and localization in crowded scenes. IEEE Trans Image Process 26(4):1992–2004
Sabokrou M, Fayyaz M, Fathy M, Moayed Z, Klette R (2018) Deep-anomaly: fully convolutional neural network for fast anomaly detection in crowded scenes. Computer Vision and Image Understanding
Tung F, Zelek JS, Clausi DA (2011) Goal-based trajectory analysis for unusual behaviour detection in intelligent surveillance. Image Vis Comput 29(4):230–240
Wang Q, Chen M, Nie F, Li X (2018) Detecting coherent groups in crowd scenes by multiview clustering. IEEE transactions on pattern analysis and machine intelligence
Wang Q, Wan J, Yuan Y (2018) Deep metric learning for crowdedness regression. IEEE Trans Circuits Syst Video Technol 28(10):2633–2643
Xiao T, Zhang C, Zha H (2015) Learning to detect anomalies in surveillance video. IEEE Signal Process Lett 22(9):1477–1481
Xu D, Song R, Wu X, Li N, Feng W, Qian H (2014) Video anomaly detection based on a hierarchical activity discovery within spatio-temporal contexts. Neurocomputing 143:144–152
Xu D, Yan Y, Ricci E, Sebe N (2017) Detecting anomalous events in videos by learning deep representations of appearance and motion. Comput Vis Image Underst 156:117–127
Yan C, Xie H, Chen J, Zha Z, Hao X, Zhang Y, Dai Q (2018) Cross-modality bridging and knowledge transferring for image understanding. IEEE Transactions on Multimedia
Yan C, Xie H, Chen J, Zha Z, Hao X, Zhang Y, Dai Q (2018) A fast uyghur text detector for complex background images. IEEE Trans Multimedia 20 (12):3389–3398. https://doi.org/10.1109/TMM.2018.2838320
Zhang Y, Lu H, Zhang L, Ruan X (2016) Combining motion and appearance cues for anomaly detection. Pattern Recogn 51:443–452
Acknowledgements
We thank Mahmudul Hasan, W.Luo, W.Liu for sharing their codes and data. This work is supported by the National Natural Science Foundation of China (Grant No. 61371192), the Key Laboratory Foundation of the Chinese Academy of Sciences (CXJJ-17S044) and the Fundamental Research Funds for the Central Universities (WK2100330002, WK3480000005).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Li, X., Li, W., Liu, B. et al. Object and patch based anomaly detection and localization in crowded scenes. Multimed Tools Appl 78, 21375–21390 (2019). https://doi.org/10.1007/s11042-019-7447-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-019-7447-1