Memory-guided two-stream spatio-temporal coding network based for video anomaly detection
Pages 826 - 832
Abstract
In the paper, we propose a memory-guided dual-stream spatio-temporal encoder network (MSTAE) based on the U-Net network as the backbone, the spatial stream uses the time displacement module to obtain the spatial features of the video, and the temporal stream is aggregated across frames to obtain the temporal features of the video, meanwhile, the coordinate attention module is introduced to improve the U-Net network and enhance the dynamic entity representation capability. In order to reduce the prediction error, the memory module is used to record the prototype patterns of normal data to reduce the problem of small error between the prediction anomaly and its true value due to the excessive generalisation ability of the deep network. We conducted extensive experiments on three publicly available standard datasets (Ped2, Avenue and ShanghaiTech datasets). The experiments demonstrate that the research model outperforms state-of-the-art methods.
References
[1]
Li N, Chang F, Liu C. Spatial-temporal cascade autoencoder for video anomaly detection in crowded scenes[J]. IEEE Transactions on Multimedia, 2020, 23: 203-215.
[2]
Chang Y, Tu Z, Xie W, Video anomaly detection with spatio-temporal dissociation[J]. Pattern Recognition, 2022, 122: 108213.
[3]
Tang Y, Zhao L, Zhang S, Integrating prediction and reconstruction for anomaly detection[J]. Pattern Recognition Letters, 2020, 129: 123-130.
[4]
Hao Y, Li J, Wang N, Spatiotemporal consistency-enhanced network for video anomaly detection[J]. Pattern Recognition, 2022, 121: 108232.
[5]
Chong Y S, Tay Y H. Abnormal event detection in videos using spatiotemporal autoencoder[C]//International symposium on neural networks. Springer, Cham, 2017: 189-196.
[6]
Doshi K, Yilmaz Y. 2021. Online anomaly detection in surveillance videos with asymptotic bound on false alarm rate. Pattern Recogn 114:107865.
[7]
Gong D, Liu L, Le V, Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 1705-1714.
[8]
Luo W, Liu W, Lian D, Video anomaly detection with sparse coding inspired deep neural networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2019, 43(3): 1070-1084.
[9]
Y.S. Chong, Y.H. Tay, Abnormal event detection in videos using spatiotemporal autoencoder, International Symposium on Neural Networks, Springer. 2017. 189–196.
[10]
Chang Y, Tu Z, Xie W, Video anomaly detection with spatio-temporal dissociation[J]. Pattern Recognition, 2022, 122: 108213.
[11]
Lin J, Gan C,Han S. Tsm: Temporal shift module for efficient video understanding[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 7083-7093.
[12]
Liu, Wen, "Future frame prediction for anomaly detection–a new baseline." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
[13]
Xie S, Guan Y. Motion instability based unsupervised online abnormal behaviors detection[J]. Multimedia Tools and Applications, 2016, 75(12): 7423-7444.
[14]
Luo W, Liu W, Gao S. A revisit of sparse coding based anomaly detection in stacked rnn framework[C]//Proceedings of the IEEE international conference on computer vision. 2017: 341-349.
[15]
Wu P, Liu J, Li M, Fast sparse coding networks for anomaly detection in videos[J]. Pattern Recognition, 2020, 107: 107515.
[16]
Park H, Noh J, Ham B. Learning memory-guided normality for anomaly detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.
[17]
Li T, Chen X, Zhu F, Two-stream deep spatial-temporal auto-encoder for surveillance video abnormal event detection[J]. Neurocomputing, 2021, 439: 256-270.
[18]
Fang Z, Zhou J T, Xiao Y, Multi-encoder towards effective anomaly detection in videos[J]. IEEE Transactions on Multimedia, 2020, 23: 4106-4116.
[19]
Nie X, Li Y, Luo L, Dynamic kernel distillation for efficient pose estimation in videos[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 6942-6950.
[20]
Deepak K, Chandrakala S, Mohan C K. Residual spatiotemporal autoencoder for unsupervised video anomaly detection[J]. Signal, Image and Video Processing, 2021, 15(1): 215-222.
[21]
Hou Q, Zhou D, Feng J. Coordqueeze-and-excitation networkstwork design[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 13713-13722.
[22]
Hu J, Shen L, Sun G 2018 Squeeze-and-excitation networks.In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141.
[23]
Woo S, Park J, Lee JY, Kweon IS 2018 Cbam: Convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19.
Index Terms
- Memory-guided two-stream spatio-temporal coding network based for video anomaly detection
Recommendations
Video anomaly detection with spatio-temporal dissociation
Highlights- We propose a novel autoencoder architecture to dissociate the spatio temporal representation and learn the regularity in both the spatial and motion feature ...
AbstractAnomaly detection in videos remains a challenging task due to the ambiguous definition of anomaly and the complexity of visual scenes from real video data. Different from the previous work which utilizes reconstruction or prediction as ...
Dual-stream spatio-temporal decoupling network for video deblurring
AbstractIt is very important to obtain spatio-temporal information in video deblurring based on deep learning. The existing methods usually jointly learn the spatio-temporal information of blurred videos through single-stream networks, which ...
Highlights- A novel dual-stream spatio-temporal decoupling network is proposed for restoring the blurred video with high quality.
Spatio-Temporal AutoEncoder for Video Anomaly Detection
MM '17: Proceedings of the 25th ACM international conference on MultimediaAnomalous events detection in real-world video scenes is a challenging problem due to the complexity of "anomaly" as well as the cluttered backgrounds, objects and motions in the scenes. Most existing methods use hand-crafted features in local spatial ...
Comments
Information & Contributors
Information
Published In
December 2023
1132 pages
ISBN:9798400716157
DOI:10.1145/3660043
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 30 May 2024
Check for updates
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICIEAI 2023
ICIEAI 2023: 2023 International Conference on Information Education and Artificial Intelligence
December 22 - 24, 2023
Xiamen, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 15Total Downloads
- Downloads (Last 12 months)15
- Downloads (Last 6 weeks)3
Reflects downloads up to 08 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format