TSF-Net:Abnormal Video Behavior Detection Framework Based on Two-Stream Appearance Motion Semantic Fusion
Pages 652 - 657
Abstract
Video anomalous behavior detection aims to detect rare and random anomalous events, which usually deviate from expectations. Existing detection methods do not consider the appearance semantics and motion semantics of the events in conjunction, so they lead to poor detection results. Since the appearance semantics and motion semantics of normal events are highly consistent and correspond, while the semantic consistency of abnormal events is low. Based on this, this paper proposes a two-stream fusion hybrid network for abnormal behavior detection. The appearance semantics and motion semantics of events are recorded separately using a multi-scale fused self-encoder to generate corresponding reconstructed appearance frames and reconstructed motion frames. Afterwards, the two frames are fused to generate the final prediction frame. We introduce temporal attention to enhance the model's ability to represent motion semantics and capture normal behavior with effective regular motion. Experiments on three standard public datasets show that the method proposed in this paper can accurately detect anomalous events with a performance comparable to state-of-the-art methods.
References
[1]
Li W, Mahadevan V, Vasconcelos N. Anomaly detection and localization in crowded scenes [J]. IEEE transactions on pattern analysis and machine intelligence, 2013, 36(1): 18-32.
[2]
Liu W, Luo W, Lian D, Future frame prediction for anomaly detection–a new baseline [C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 6536-6545.
[3]
Wu A, Han Y. Multi-modal Circulant Fusion for Video-to-Language and Backward [C]//IJCAI. 2018, 3(4): 8.Gong D, Liu L, Le V, Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 1705-1714.
[4]
Yang J, Li C, Dai X, Focal modulation networks [J]. Advances in Neural Information Processing Systems, 2022, 35: 4203-4217.
[5]
Zhu C, Chen F, Ahmed U,et al. Semantic relation reasoning for shot-stable few-shot object detection [C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 8782-8791.
[6]
Hu J, Shen L, Sun G. Squeeze-and-excitation networks [C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 7132-7141.
[7]
Carreira J, Zisserman A. Quo vadis, action recognition? a new model and the kinetics dataset [C]//proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 6299-6308.
[8]
Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation [C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2015: 3431-3440.
[9]
Zhao B, Fei-Fei L, Xing E P. Online detection of unusual events in videos via dynamic sparse coding [C]//CVPR 2011. IEEE, 2011: 3313-3320.
[10]
Lu C, Shi J, Jia J. Abnormal event detection at 150 fps in matlab [C]//Proceedings of the IEEE international conference on computer vision. 2013: 2720-2727.
[11]
Y. Cong, J. Yuan, and J. Liu, ‘‘Sparse reconstruction cost for abnormal event detection,’’ in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2011, pp. 3449–3456.
[12]
Gong D, Liu L, Le V, Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 1705-1714.
[13]
Mahadevan V, Li W, Bhalodia V, Anomaly detection in crowded scenes [C]//2010 IEEE computer society conference on computer vision and pattern recognition. IEEE, 2010: 1975-1981.
[14]
Lu C, Shi J, Jia J. Abnormal event detection at 150 fps in matlab [C]//Proceedings of the IEEE international conference on computer vision. 2013: 2720-2727.
[15]
Luo W, Liu W, Gao S. A revisit of sparse coding based anomaly detection in stacked rnn framework [C]//Proceedings of the IEEE international conference on computer vision. 2017: 341-349.
[16]
Zhao Y, Deng B, Shen C, Spatio-temporal autoencoder for video anomaly detection [C]//Proceedings of the 25th ACM international conference on Multimedia. 2017: 1933-1941.
[17]
W, Liu W, Lian D, Future frame prediction network for video anomaly detection [J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 44(11): 7505-7520.
[18]
Zhang Q, Feng G, Wu H. Surveillance video anomaly detection via non-local U-Net frame prediction [J]. Multimedia Tools and Applications, 2022, 81(19): 27073-27088.
[19]
Zhou J T, Du J, Zhu H, Anomalynet: An anomaly detection network for video surveillance [J]. IEEE Transactions on Information Forensics and Security, 2019, 14(10): 2537-2550.
[20]
Zhian Liu, Yongwei Nie, Chengjiang Long, Qing Zhang, and Guiqing Li. A hybrid video anomaly detection framework via memory-augmented flow reconstruction and flowguided frame prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 13588–13597, October 2021.
[21]
Chang Y, Tu Z, Xie W, Clustering driven deep autoencoder for video anomaly detection [C]//European Conference on Computer Vision. Springer, Cham, 2020: 329-345.
[22]
Bowen Liu, Yu Chen, Shiyu Liu, and Hun-Seok Kim. Deep learning in latent space for video prediction and compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 701–710, 2021.
[23]
Yunpeng Chang, Zhigang Tu, Wei Xie, Bin Luo, Shifu Zhang, Haigang Sui, and Junsong Yuan. Video anomaly detection with spatio-temporal dissociation. Pattern Recognition, 122:108213, 2022.
[24]
Hao Y, Li J, Wang N, Spatiotemporal consistency-enhanced network for video anomaly detection [J]. Pattern Recognition, 2022, 121: 108232.
[25]
Liu B, Chen Y, Liu S, Deep learning in latent space for video prediction and compression [C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 701-710.
[26]
Yi Hao, Jie Li, Nannan Wang, Xiaoyu Wang, and Xinbo Gao. Spatiotemporal consistency-enhanced network for video anomaly detection. Pattern Recognition, 121:108232, 2022.
[27]
Trong-Nguyen Nguyen and Jean Meunier. Anomaly detection in video sequence with appearance-motion correspondence. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1273–1283, 2019.
[28]
Liu Y, Liu J, Yang K, Amp-net: Appearance-motion prototype network assisted automatic video anomaly detection system [J]. IEEE Transactions on Industrial Informatics, 2023.
[29]
Cai R, Zhang H, Liu W, Appearance-motion memory consistency network for video anomaly detection [C]//Proceedings of the AAAI conference on artificial intelligence. 2021, 35(2): 938-946.
Index Terms
- TSF-Net:Abnormal Video Behavior Detection Framework Based on Two-Stream Appearance Motion Semantic Fusion
Recommendations
Appearance-Motion Fusion Network for Video Anomaly Detection
Pattern Recognition and Computer VisionAbstractDetection of abnormal events in surveillance video is an important and challenging task, which has received much research interest over the past few years. However, existing methods often only considered appearance information or simply integrated ...
Abnormal event detection for video surveillance using an enhanced two-stream fusion method
Graphical abstractDisplay Omitted
Highlights- We proposed a unified classification and prediction fusion framework for detecting different types of abnormal events in surveillance videos.
- With the assistance of motion features, our AED algorithm is more sensitive to abnormal ...
AbstractAbnormal event detection is a critical component of intelligent surveillance systems, focusing on identifying abnormal objects or unusual human behaviours in video sequences. However, conventional methods struggle due to the scarcity of labelled ...
Video anomaly detection and localization based on appearance and motion models
AbstractIn this paper, we present an approach to detect and localize anomalies in the surveillance videos. Precise detection, modeling the normality in a context and dealing with false alarms are the major challenges to cope with while performing ...
Comments
Information & Contributors
Information
Published In
December 2023
1132 pages
ISBN:9798400716157
DOI:10.1145/3660043
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 30 May 2024
Check for updates
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICIEAI 2023
ICIEAI 2023: 2023 International Conference on Information Education and Artificial Intelligence
December 22 - 24, 2023
Xiamen, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 19Total Downloads
- Downloads (Last 12 months)19
- Downloads (Last 6 weeks)4
Reflects downloads up to 08 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format