research-article

Detecting Abnormality without Knowing Normality: A Two-stage Approach for Unsupervised Video Abnormal Event Detection

Authors:

Chengzhang Zhu,

Jianping YinAuthors Info & Claims

MM '18: Proceedings of the 26th ACM international conference on Multimedia

Pages 636 - 644

https://doi.org/10.1145/3240508.3240615

Published: 15 October 2018 Publication History

Abstract

Abnormal event detection in video surveillance is a valuable but challenging problem. Most methods adopt a supervised setting that requires collecting videos with only normal events for training. However, very few attempts are made under unsupervised setting that detects abnormality without priorly knowing normal events. Existing unsupervised methods detect drastic local changes as abnormality, which overlooks the global spatio-temporal context. This paper proposes a novel unsupervised approach, which not only avoids manually specifying normality for training as supervised methods do, but also takes the whole spatio-temporal context into consideration. Our approach consists of two stages: First, normality estimation stage trains an autoencoder and estimates the normal events globally from the entire unlabeled videos by a self-adaptive reconstruction loss thresholding scheme. Second, normality modeling stage feeds the estimated normal events from the previous stage into one-class support vector machine to build a refined normality model, which can further exclude abnormal events and enhance abnormality detection performance. Experiments on various benchmark datasets reveal that our method is not only able to outperform existing unsupervised methods by a large margin (up to 14.2% AUC gain), but also favorably yields comparable or even superior performance to state-of-the-art supervised methods.

References

[1]

Amit Adam, Ehud Rivlin, Ilan Shimshoni, and Daviv Reinitz. 2008. Robust real-time unusual event detection using multiple fixed-location monitors. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Vol. 30, 3 (2008), 555--560.

Digital Library

[2]

Arslan Basharat, Alexei Gritai, and Mubarak Shah. 2008. Learning object motion patterns for anomaly detection and improved object detection. In Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 1--8.

[3]

Marco Bertini, Alberto Del Bimbo, and Lorenzo Seidenari. 2012. Multi-scale and real-time non-parametric approach for anomaly detection and localization. Computer Vision and Image Understanding, Vol. 116, 3 (2012), 320--329.

Digital Library

[4]

Varun Chandola, Arindam Banerjee, and Vipin Kumar. 2009. Anomaly detection: A survey. ACM computing surveys (CSUR), Vol. 41, 3 (2009), 15.

Digital Library

[5]

Pau-Choo Chung and Chin-De Liu. 2008. A daily behavior enabled hidden Markov model for human behavior understanding. Pattern Recognition, Vol. 41, 5 (2008), 1572--1580.

Digital Library

[6]

Yang Cong, Junsong Yuan, and Ji Liu. 2011. Sparse reconstruction cost for abnormal event detection. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 3449--3456.

Digital Library

[7]

Yang Cong, Junsong Yuan, and Yandong Tang. 2013. Video anomaly search in crowded scenes via spatio-temporal motion context. IEEE Transactions on Information Forensics and Security (TIFS), Vol. 8, 10 (2013), 1590--1599.

Digital Library

[8]

Allison Del Giorno, J Andrew Bagnell, and Martial Hebert. 2016. A discriminative framework for anomaly detection in large videos. In Proceedings of the 2016 European Conference on Computer Vision (ECCV). Springer, 334--349.

[9]

Homa Foroughi, Alireza Rezvanian, and Amirhossien Paziraee. 2008. Robust fall detection using human shape and multi-class support vector machine. In Proceedings of the 2008 Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP). IEEE, 413--420.

Digital Library

[10]

Zhouyu Fu, Weiming Hu, and Tieniu Tan. 2005. Similarity based vehicle trajectory clustering and anomaly detection. In Proceedings of the 2005 IEEE International Conference on Image Processing (ICIP), Vol. 2. IEEE, II--602.

[11]

Mahmudul Hasan, Jonghyun Choi, Jan Neumann, Amit K Roy-Chowdhury, and Larry S Davis. 2016. Learning temporal regularity in video sequences. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 733--742.

[12]

Jaechul Kim and Kristen Grauman. 2009. Observe locally, infer globally: a space-time MRF for detecting abnormal activities with incremental updates. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2921--2928.

[13]

Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 2015 International Conference on Learning Representations (ICLR) .

[14]

Louis Kratz and Ko Nishino. 2009. Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 1446--1453.

[15]

Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. 1998. Gradient-Based Learning Applied to Document Recognition. Proc. IEEE, Vol. 86, 11 (November 1998), 2278--2324.

[16]

Weixin Li, Vijay Mahadevan, and Nuno Vasconcelos. 2014a. Anomaly detection and localization in crowded scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Vol. 36, 1 (2014), 18--32.

Digital Library

[17]

Yuanwei Li, En Zhu, Jianmin Zhao, Jianping Yin, and Xiangfu Zhao. 2014b. A fast simple optical flow computation approach based on the 3-D gradient. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 24, 5 (2014), 842--853.

[18]

Hanhe Lin, Jeremiah D Deng, Brendon J Woodford, and Ahmad Shahi. 2016. Online weighted clustering for real-time abnormal event detection in video surveillance. In Proceedings of the 2016 ACM on Multimedia Conference (MM). ACM, 536--540.

Digital Library

[19]

W. Liu, D. Lian W. Luo, and S. Gao. 2018. Future Frame Prediction for Anomaly Detection -- A New Baseline. In 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) .

[20]

Cewu Lu, Jianping Shi, and Jiaya Jia. 2013. Abnormal event detection at 150 fps in matlab. In Proceedings of the 2013 IEEE International Conference on Computer Vision (ICCV). IEEE, 2720--2727.

Digital Library

[21]

Weixin Luo, Wen Liu, and Shenghua Gao. 2017. A Revisit of Sparse Coding Based Anomaly Detection in Stacked RNN Framework. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (ICCV) . 341--349.

[22]

Vijay Mahadevan, Weixin Li, Viral Bhalodia, and Nuno Vasconcelos. 2010. Anomaly detection in crowded scenes. In Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 1975--1981.

[23]

Ramin Mehran, Alexis Oyama, and Mubarak Shah. 2009. Abnormal crowd behavior detection using social force model. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 935--942.

[24]

Claudio Piciarelli, Christian Micheloni, and Gian Luca Foresti. 2008. Trajectory-based anomalous event detection. IEEE Transactions on Circuits and Systems for video Technology, Vol. 18, 11 (2008), 1544--1554.

Digital Library

[25]

Bernhard Schölkopf, John C Platt, John Shawe-Taylor, Alex J Smola, and Robert C Williamson. 2001. Estimating the support of a high-dimensional distribution. Neural computation, Vol. 13, 7 (2001), 1443--1471.

Digital Library

[26]

Angela A Sodemann, Matthew P Ross, and Brett J Borghetti. 2012. A review of anomaly detection in automated surveillance. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), Vol. 42, 6 (2012), 1257--1272.

Digital Library

[27]

Hanh TM Tran and DC Hogg. 2017. Anomaly Detection using a Convolutional Winner-Take-All Autoencoder. In Proceedings of the 2017 British Machine Vision Conference (BMVC). BMVA Press.

[28]

Radu Tudor Ionescu, Sorina Smeureanu, Bogdan Alexe, and Marius Popescu. 2017. Unmasking the abnormal events in video. In Proceedings of the 2017 IEEE Conference on Computer Vision (ICCV). 2895--2903.

[29]

Siqi Wang, En Zhu, Jianping Yin, and Fatih Porikli. 2016. Anomaly detection in crowded scenes by SL-HOF descriptor and foreground classification. In Proceedings of the 2016 International Conference on Pattern Recognition (ICPR). IEEE, 3398--3403.

[30]

D Xu, J Song, Y Yan, E Ricci, N Sebe, et almbox. 2015. Learning Deep Representations of Appearance and Motion for Anomalous Event Detection. In Proceedings of the 2015 British Machine Vision Conference (BMVC). BMVA Press.

[31]

Yiru Zhao, Bing Deng, Chen Shen, Yao Liu, Hongtao Lu, and Xian-Sheng Hua. 2017. Spatio-Temporal AutoEncoder for Video Anomaly Detection. In Proceedings of the 2017 ACM on Multimedia Conference (MM). ACM, 1933--1941.

Digital Library

Cited By

Sharif MJiao LOmlin C(2025)Deep crowd anomaly detection: state-of-the-art, challenges, and future research directionsArtificial Intelligence Review10.1007/s10462-024-11092-858:5Online publication date: 20-Feb-2025
https://doi.org/10.1007/s10462-024-11092-8
Zhang MWang JQi QRen PSun HZhuang ZZhang LLiao JLarson K(2024)Safeguarding sustainable citiesProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/838(7572-7580)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.24963/ijcai.2024/838
Wu PZhou XPang GYang ZYan QWang PZhang YCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Weakly Supervised Video Anomaly Detection and Localization with Spatio-Temporal PromptsProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681442(9301-9310)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681442
Show More Cited By

Index Terms

Detecting Abnormality without Knowing Normality: A Two-stage Approach for Unsupervised Video Abnormal Event Detection
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Scene anomaly detection
  2. Machine learning
    1. Learning paradigms
      1. Unsupervised learning

Recommendations

Feasibility of Universal Anomaly Detection Without Knowing the Abnormality in Medical Images
Medical Image Learning with Limited and Noisy Data
Abstract
Many anomaly detection approaches, especially deep learning methods, have been recently developed to identify abnormal image morphology by only employing normal images during training. Unfortunately, many prior anomaly detection methods were ...
Few-Shot Action Localization without Knowing Boundaries
ICMR '21: Proceedings of the 2021 International Conference on Multimedia Retrieval

Learning to localize actions in long, cluttered, and untrimmed videos is a hard task, that in the literature has typically been addressed assuming the availability of large amounts of annotated training samples for each class -- either in a fully-...
Active Learning without Knowing Individual Instance Labels: A Pairwise Label Homogeneity Query Approach

Traditional active learning methods require the labeler to provide a class label for each queried instance. The labelers are normally highly skilled domain experts to ensure the correctness of the provided labels, which in turn results in expensive ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '18: Proceedings of the 26th ACM international conference on Multimedia

October 2018

2167 pages

ISBN:9781450356657

DOI:10.1145/3240508

General Chairs:
Susanne Boll
University of Oldenburg, Germany
,
Kyoung Mu Lee
Seoul National University, Korea
,
Jiebo Luo
University of Rochester, USA
,
Wenwu Zhu
Tsinghua University, China
,
Program Chairs:
Hyeran Byun
Yonsei University, Korea
,
Chang Wen Chen
State Univ. Of New York at Buffalo, USA
,
Rainer Lienhart
University of Augsburg, Germany
,
Tao Mei
JD AI, China

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China
National Key R&D Program of China

Conference

MM '18

Sponsor:

SIGMM

MM '18: ACM Multimedia Conference

October 22 - 26, 2018

Seoul, Republic of Korea

Acceptance Rates

MM '18 Paper Acceptance Rate 209 of 757 submissions, 28%;

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

43
Total Citations
View Citations
793
Total Downloads

Downloads (Last 12 months)44
Downloads (Last 6 weeks)13

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Sharif MJiao LOmlin C(2025)Deep crowd anomaly detection: state-of-the-art, challenges, and future research directionsArtificial Intelligence Review10.1007/s10462-024-11092-858:5Online publication date: 20-Feb-2025
https://doi.org/10.1007/s10462-024-11092-8
Zhang MWang JQi QRen PSun HZhuang ZZhang LLiao JLarson K(2024)Safeguarding sustainable citiesProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/838(7572-7580)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.24963/ijcai.2024/838
Wu PZhou XPang GYang ZYan QWang PZhang YCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Weakly Supervised Video Anomaly Detection and Localization with Spatio-Temporal PromptsProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681442(9301-9310)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681442
Pi RWu PHe XPeng Y(2024)EOGT: Video Anomaly Detection with Enhanced Object Information and Global Temporal DependencyACM Transactions on Multimedia Computing, Communications, and Applications10.1145/366218520:10(1-21)Online publication date: 12-Sep-2024
https://dl.acm.org/doi/10.1145/3662185
Shen GOuyang YLu JYang YSanchez V(2024)Advancing Video Anomaly Detection: A Bi-Directional Hybrid Framework for Enhanced Single- and Multi-Task ApproachesIEEE Transactions on Image Processing10.1109/TIP.2024.351236933(6865-6880)Online publication date: 2024
https://doi.org/10.1109/TIP.2024.3512369
Wang ZGu XGu X(2024)Transformer Based Sptial-Temporal Extraction Model for Video Anomaly Detection2024 8th International Conference on Robotics, Control and Automation (ICRCA)10.1109/ICRCA60878.2024.10649355(370-374)Online publication date: 12-Jan-2024
https://doi.org/10.1109/ICRCA60878.2024.10649355
Hong SAhn SJo YPark S(2024)Making Anomalies More Anomalous: Video Anomaly Detection Using a Novel Generator and DestroyerIEEE Access10.1109/ACCESS.2024.337438312(36712-36726)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3374383
Guo BLiu MHe QJiang M(2024)Two-Stream Spatial-Temporal Auto-Encoder With Adversarial Training for Video Anomaly DetectionIEEE Access10.1109/ACCESS.2023.327164712(125881-125889)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2023.3271647
Merlin RKarthick RBabu ASelvi GUsha DNithya R(2024)Abnormal events detection using spatio-temporal saliency descriptor and fuzzy representation analysisScientific Reports10.1038/s41598-024-81387-x14:1Online publication date: 30-Nov-2024
https://doi.org/10.1038/s41598-024-81387-x
Tao YHu YChen Z(2024)Memory-guided representation matching for unsupervised video anomaly detectionJournal of Visual Communication and Image Representation10.1016/j.jvcir.2024.104185101(104185)Online publication date: May-2024
https://doi.org/10.1016/j.jvcir.2024.104185
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten