Skip to main content
SpringerLink
Log in

Detecting Anomalies in Videos using Perception Generative Adversarial Network

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

This paper presents a novel end-to-end unsupervised deep learning approach for video anomaly detection. We propose to utilize the Perception Generative Adversarial Net (Perception-GAN), which is trained using the initial portion of the video. The generator of the perceptual-GAN learns how to generate events similar to the normal events, while the discriminator of the perceptual-GAN learns how to distinguish the abnormal events from the normal events. At testing time, only the discriminator is used to solve our discriminative task (abnormality detection). Through combining the generative adversarial loss and the proposed perceptual adversarial loss, these two networks can be trained alternately to solve the anomaly detection task. A two-stream networks framework and an update strategy is employed to improve the detection result. We test our approach on three popular benchmarks and the experimental results verify the superiority of our method compared to the state of the arts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

The datasets analyzed during the current study are available in the repository [UCSD datasets: http://www.svcl.ucsd.edu/projects/anomaly/dataset.htm. UMN datasets: http://mha.cs.umn.edu/Movies/Crowd-Activity-All.avi. Subway datasets: http://vision.eecs.yorku.ca/research/anomalous-behaviour-data/sets/Amit-Subway.zip

References

  1. A. Adam, E. Rivlin, I. Shimshoni, D. Reinitz, Robust real-time unusual event detection using multiple fixed location monitors. IEEE Trans. Pattern Anal. Mach. Intell. 30, 555–560 (2008)

    Article  Google Scholar 

  2. V. Chandola, A. Banerjee, V. Kumar, Anomaly detection: a survey. ACM Comput. Surv. 41(3), 1–72 (2009)

    Article  Google Scholar 

  3. R. Chaker, Z.A. Aghbari, I.N. Junejo, Social network model for crowd anomaly detection and localization. Pattern Recogn. 61, 266–281 (2017)

    Article  Google Scholar 

  4. N. Dilokthanakul, P. A. M. Mediano, M. Garnelo, M. C. H. Lee, H. Salimbeni, K. Arulkumaran and M. Shanahan, Deep unsupervised clustering with gaussian mixture variational autoencoders. In International Conference on Learning Representations (2017)

  5. J. K. Dutta, B. Banerjee, Online detection of abnormal events using incremental coding length. In AAAI Conference on Artificial Intelligence, pp. 3755–3761 (2015)

  6. A. D. Giorno, J. A. Bagnell, M. Hebert, A discriminative framework for anomaly detection in large videos. In European Conference on Computer Vision, (2016)

  7. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D.Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial nets. In International Conference on Neural Information Processing Systems (2014)

  8. Half-Cooked Research Reports, Video Surveillance Storage Market Research Report - Forecast to 2023 (2019)

  9. M. Hasan, J. Choi. J. Neumanny, A. K. Roy-Chowdhury, L. S. Davis, Learning temporal regularity in video sequences. In IEEE Conference on Computer Vision and Pattern Recognition (2016)

  10. Y. Hu, Y. Zhang, L. S. Davis, Unsupervised abnormal crowd activity detection using semiparametric scan statistic. In IEEE Conference on Computer Vision and Pattern Recognition Workshops (2013)

  11. H. Huang, P. S. Yu, C. Wang, An introduction to image synthesis with generative adversarial nets. https://arxiv.org/abs/1803.04469

  12. R. T. Ionescu, S. Smeureanu, B. Alexe, M. Popescu, Unmasking the abnormal events in video. In International Conference on Computer Vision (2017)

  13. P. Isola, J. Zhu, T. Zhou, A. A. Efros, Image-to-image translation with conditional adversarial networks. In IEEE Conference on Computer Vision and Pattern Recognition (2017)

  14. Y. Ito, K. M. Kitani, J. A. Bagnell, M. Hebert, Detecting interesting events using unsupervised density ratio estimation. In European Conference on Computer Vision Workshop, (2012)

  15. J. Johnson, A. Alahi, L. Fei-Fei, Perceptual losses for real-time style transfer and super-resolution. In European Conference on Computer Vision, pp. 694–711 (2016)

  16. J. Kim and K Grauman, Observe locally, infer globally: a space-time mrf for detecting abnormal activities with incremental updates. In IEEE Conference on Computer Vision and Pattern Recognition (2009)

  17. D. P. Kingma, M. Welling, Auto-encoding variational bayes. In International Conference on Learning Representations (2014)

  18. W. Li, V. Mahadevan, N. Vasconcelos, Anomaly detection and localization in crowded scenes. IEEE Trans. Pattern Anal. Mach. Intell. 36(1), 18–32 (2014)

    Article  Google Scholar 

  19. J. Li, X. Liang, Y. Wei, T. Xu, J. Feng, S. Yan, Perceptual generative adversarial networks for small object detection. In IEEE Conference on Computer Vision and Pattern Recognition (2017)

  20. C. Liu, Beyond pixels: exploring new representations and applications for motion analysis. Massachusetts Institute of Technology (2009).

  21. Y. Liu, C. Li, B. P oczos, Classifier two sample test for video anomaly detections. British Machine Vision Conference (2018)

  22. Y. Lu, F. Yu, M. K. K. Reddy, Y. Wang, Few-shot scene-adaptive anomaly detection. European Conference on Computer Vision (2020)

  23. V. Mahadevan, W. Li, V. Bhalodia, N. Vasconcelos, Anomaly detection in crowded scenes,in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1975–1981 (2010).

  24. R. Mehran, A. Oyama, M. Shah, Abnormal Crowd Behavior Detection using Social Force Model. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 935–942 (2009)

  25. G. Pang, C. Yan, C. Shen, A. v. d. Hengel, X. Bai, Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detection. IEEE Conference on Computer Vision and Pattern Recognition (2020)

  26. M. Ravanbakhsh, M. Nabi, E. Sangineto, L. Marcenaro, C. Regazzoni and N. Sebe, Abnormal event detection in videos using generative adversarial nets. In IEEE International Conference on Image Processing (2018)

  27. M.J. Roshtkhari, M.D. Levine, An on-line, real-time learning method for detecting anomalies in videos using spatio-temporal compositions. Comput. Vis. Image Underst. 117(10), 1436–1452 (2013)

    Article  Google Scholar 

  28. M. J. Roshtkhari, M. D. Levine, Online dominant and anomalous behavior detection in videos. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 2611–2618 (2013)

  29. M. Sabokrou, M. Fathy, M. Hoseini, Video anomaly detection and localization based on the sparsity and reconstruction error of autoencoder. IET Electron. Lett. 52(13), 1122–1124 (2016)

    Article  Google Scholar 

  30. M. Sabokrou, M. Fayyaz, M. Fathy, R. Klette, Deep-cascade: cascading 3D deep neural networks for fast anomaly detection and localization in crowded scenes. IEEE Trans. Image Process. 26(4), 1992–2004 (2017)

    Article  MathSciNet  Google Scholar 

  31. K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations (2015)

  32. Q. Sun, H. Liu, T. Harada, Online growing neural gas for anomaly detection in changing surveillance scenes. Pattern Recogn. 64, 187–201 (2017)

    Article  Google Scholar 

  33. D. Sungatullina, E. Zakharov, D. Ulyanov, V. Lempitsky, Image manipulation with perceptual discriminators. In European Conference on Computer Vision (2018)

  34. X. Tang, S. Zhang, H. Yao, Sparse coding based motion attention for abnormal event detection. In International Conference on Image Processing (2013)

  35. D. Xu, Y. Yan, E. Ricci, N. Sebe, Detecting anomalous events in videos by learning deep representations of appearance and motion. Comput. Vis. Image Underst. 156, 117–127 (2017)

    Article  Google Scholar 

  36. B. Zhao, L. Fei-Fei, E. P. Xing, Online detection of unusual events in videos via dynamic sparse coding. In IEEE Conference on Computer Vision and Pattern Recognition (2011)

  37. Y. Zhang, H. Lu, L. Zhang, X. Ruan, Combining motion and appearance cues for anomaly detection. Pattern Recogn. 51, 443–452 (2016)

    Article  Google Scholar 

  38. J. Zhu, T. Park, P. Isola and A. A. Efros, Unpaired image-to-image translation using cycle-consistent adversarial networks. In IEEE International Conference on Computer Vision (2018)

Download references

Author information

Authors and Affiliations

  1. National Key Laboratory of Science and Technology On Vessel Integrated Power System, Naval University of Engineering, Wuhan, 430033, China

    Yaxiang Fan, Fei Xiao & Shaohua Qiu

  2. Science and Technology On Automatic Target Recognition Laboratory (ATR), National University of Defense Technology, Changsha, 410073, China

    Yaxiang Fan, Gongjian Wen & Shaohua Qiu

  3. State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, 430071, Hubei, China

    Deren Li

Authors
  1. Yaxiang Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gongjian Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fei Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shaohua Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Deren Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fei Xiao.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fan, Y., Wen, G., Xiao, F. et al. Detecting Anomalies in Videos using Perception Generative Adversarial Network. Circuits Syst Signal Process 41, 994–1018 (2022). https://doi.org/10.1007/s00034-021-01820-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-021-01820-8

Keywords

Search

Navigation