Skip to main content
SpringerLink
Log in
Book cover

International Conference on Computer Vision Systems

ICVS 2019: Computer Vision Systems pp 330–342Cite as

Detecting Video Anomaly with a Stacked Convolutional LSTM Framework

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11754))

Abstract

Automatic anomaly detection in real-world video surveillance is still challenging. In this paper, we propose an autoencoder architecture based on a stacked convolutional LSTM framework that highlights both spatial and temporal aspects in detecting anomalies of surveillance videos. The spatial component(i.e. spatial encoder/decoder) uses Convolutional Neural Network (CNN) and carries information about scenes and objects. The temporal component(i.e. temporal encoder/decoder) uses stacked convolutional LSTM and conveys object movement. Specifically, we integrate CNN and the stacked convolutional LSTM to learn normal patterns from the training data, which contains only normal events. With the integrated approach, our method can better model spatio-temporal information than many others. We train our models in an unsupervised manner, and labels are required only in the testing phase. Our method is evaluated on the datasets of Avenue, UCSD and ShanghaiTech Campus. The results show that the accuracy of our method rivals state-of-the-art methods with a faster detection speed.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Adam, A., Rivlin, E., Shimshoni, I., Reinitz, D.: Robust real-time unusual event detection using multiple fixed-location monitors. IEEE Trans. Pattern Anal. Mach. Intell. 30(3), 555–560 (2008). https://doi.org/10.1109/TPAMI.2007.70825

    Article  Google Scholar 

  2. Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. (CSUR) 41(3), 15 (2009). https://doi.org/10.1145/1541880.1541882

    Article  Google Scholar 

  3. Chong, Y.S., Tay, Y.H.: Abnormal event detection in videos using spatiotemporal autoencoder. In: Cong, F., Leung, A., Wei, Q. (eds.) ISNN 2017. LNCS, vol. 10262, pp. 189–196. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59081-3_23

    Chapter  Google Scholar 

  4. Cong, Y., Yuan, J., Liu, J.: Sparse reconstruction cost for abnormal event detection. In: CVPR 2011, pp. 3449–3456 (2011). https://doi.org/10.1109/CVPR.2011.5995434

  5. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Schmid, C., Soatto, S., Tomasi, C. (eds.) International Conference on Computer Vision & Pattern Recognition (CVPR 2005), vol. 1, pp. 886–893. IEEE Computer Society, San Diego, June 2005. https://doi.org/10.1109/CVPR.2005.177

  6. Dalal, N., Triggs, B., Schmid, C.: Human detection using oriented histograms of flow and appearance. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 428–441. Springer, Heidelberg (2006). https://doi.org/10.1007/11744047_33

    Chapter  Google Scholar 

  7. Zhang, D., Gatica-Perez, D., Bengio, S., McCowan, I.: Semi-supervised adapted hmms for unusual event detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 1, pp. 611–618, June 2005. https://doi.org/10.1109/CVPR.2005.316

  8. Girshick, R.: Fast r-CNN. In: 2015 IEEE International Conference on Computer Vision (ICCV). IEEE, December 2015. https://doi.org/10.1109/iccv.2015.169

  9. Hasan, M., Choi, J., Neumann, J., Roy-Chowdhury, A.K., Davis, L.S.: Learning temporal regularity in video sequences. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Jun 2016. https://doi.org/10.1109/cvpr.2016.86

  10. Kim, J., Grauman, K.: Observe locally, infer globally: a space-time MRF for detecting abnormal activities with incremental updates. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2928, June 2009. https://doi.org/10.1109/CVPR.2009.5206569

  11. Kratz, L., Nishino, K.: Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1446–1453, June 2009. https://doi.org/10.1109/CVPR.2009.5206771

  12. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Association for Computing Machinery (ACM), vol. 60, pp. 84–90, May 2017. https://doi.org/10.1145/3065386

    Article  Google Scholar 

  13. Li, C., Han, Z., Ye, Q., Jiao, J.: Abnormal behavior detection via sparse reconstruction analysis of trajectory. In: 2011 Sixth International Conference on Image and Graphics, pp. 807–810, August 2011. https://doi.org/10.1109/ICIG.2011.104

  14. Liu, W., Luo, W., Lian, D., Gao, S.: Future frame prediction for anomaly detection - a new baseline. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, June 2018.https://doi.org/10.1109/cvpr.2018.00684

  15. Lu, C., Shi, J., Jia, J.: Abnormal event detection at 150 fps in matlab. In: 2013 IEEE International Conference on Computer Vision, pp. 2720–2727, December 2013. https://doi.org/10.1109/ICCV.2013.338

  16. Luo, W., Liu, W., Gao, S.: Remembering history with convolutional lstm for anomaly detection. In: 2017 IEEE International Conference on Multimedia and Expo (ICME), pp. 439–444, July 2017. https://doi.org/10.1109/ICME.2017.8019325

  17. Luo, W., Liu, W., Gao, S.: A revisit of sparse coding based anomaly detection in stacked RNN framework. In: 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, October 2017. https://doi.org/10.1109/iccv.2017.45

  18. Mahadevan, V., Li, W., Bhalodia, V., Vasconcelos, N.: Anomaly detection in crowded scenes. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1975–1981, June 2010. https://doi.org/10.1109/CVPR.2010.5539872

  19. Medel, J.R.: Anomaly detection using predictive convolutional long short-term memory units (2016)

    Google Scholar 

  20. Piciarelli, C., Micheloni, C., Foresti, G.L.: Trajectory-based anomalous event detection. IEEE Trans. Circ. Syst. Video Technol. 18(11), 1544–1554 (2008). https://doi.org/10.1109/TCSVT.2008.2005599

    Article  Google Scholar 

  21. Reddy, V., Sanderson, C., Lovell, B.C.: Improved anomaly detection in crowded scenes via cell-based analysis of foreground speed, size and texture. In: CVPR 2011 WORKSHOPS, pp. 55–61, June 2011. https://doi.org/10.1109/CVPRW.2011.5981799

  22. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-CNN: Towards real-time object detection with region proposal networks, vol. 39, pp. 1137–1149. Institute of Electrical and Electronics Engineers (IEEE), June 2017. https://doi.org/10.1109/tpami.2016.2577031

    Article  Google Scholar 

  23. Sabokrou, M., Fathy, M., Hoseini, M., Klette, R.: Real-time anomaly detection and localization in crowded scenes. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 56–62, June 2015. https://doi.org/10.1109/CVPRW.2015.7301284

  24. Shi, X., Chen, Z., Wang, H., Yeung, D.Y., Wong, W.K., Woo, W.C.: Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems 28, pp. 802–810. Curran Associates Inc., New York (2015). http://papers.nips.cc/paper/5955-convolutional-lstm-network-a-machine-learning-approach-for-precipitation-nowcasting.pdf

    Google Scholar 

  25. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  26. Sultani, W., Chen, C., Shah, M.: Real-world anomaly detection in surveillance videos. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Jun 2018. https://doi.org/10.1109/cvpr.2018.00678

  27. Tung, F., Zelek, J.S., Clausi, D.A.: Goal-based trajectory analysis for unusual behaviour detection in intelligent surveillance. Image Vis. Comput. 29(4), 230–240 (2011). https://doi.org/10.1016/j.imavis.2010.11.003

    Article  Google Scholar 

  28. Wu, S., Moore, B.E., Shah, M.: Chaotic invariants of lagrangian particle trajectories for anomaly detection in crowded scenes. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2054–2060, June 2010. https://doi.org/10.1109/CVPR.2010.5539882

  29. Xu, D., Ricci, E., Yan, Y., Song, J., Sebe, N.: Learning deep representations of appearance and motion for anomalous event detection (2015). https://doi.org/10.5244/c.29.8

  30. Yen, S., Wang, C.: Abnormal event detection using HOSF. In: 2013 International Conference on IT Convergence and Security (ICITCS), pp. 1–4, December 2013. https://doi.org/10.1109/ICITCS.2013.6717798

  31. Zhao, B., Fei-Fei, L., Xing, E.P.: Online detection of unusual events in videos via dynamic sparse coding. CVPR 2011, 3313–3320 (2011). https://doi.org/10.1109/CVPR.2011.5995524

    Article  Google Scholar 

  32. Zhou, S., Shen, W., Zeng, D., Zhang, Z.: Unusual event detection in crowded scenes by trajectory analysis. In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1300–1304, April 2015. https://doi.org/10.1109/ICASSP.2015.7178180

Download references

Author information

Authors and Affiliations

  1. Institute of Software Chinese Academy of Sciences, Beijing, China

    Hao Wei, Kai Li, Haichang Li, Yifan Lyu & Xiaohui Hu

Authors
  1. Hao Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kai Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haichang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yifan Lyu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaohui Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hao Wei .

Editor information

Editors and Affiliations

  1. Centre for Research and Technology Hellas (CERTH-ITI), Thessaloniki, Greece

    Dimitrios Tzovaras

  2. Centre for Research and Technology Hellas (CERTH-ITI), Thessaloniki, Greece

    Dimitrios Giakoumis

  3. Vienna University of Technology, Vienna, Austria

    Markus Vincze

  4. Foundation for Research and Technology Hellas (FORTH), Heraklion, Greece

    Antonis Argyros

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wei, H., Li, K., Li, H., Lyu, Y., Hu, X. (2019). Detecting Video Anomaly with a Stacked Convolutional LSTM Framework. In: Tzovaras, D., Giakoumis, D., Vincze, M., Argyros, A. (eds) Computer Vision Systems. ICVS 2019. Lecture Notes in Computer Science(), vol 11754. Springer, Cham. https://doi.org/10.1007/978-3-030-34995-0_30

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-34995-0_30

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-34994-3

  • Online ISBN: 978-3-030-34995-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation