Air-CAD: Edge-Assisted Multi-Drone Network for Real-time Crowd Anomaly Detection
Pages 2817 - 2825
Abstract
Drones connected via the web are increasingly being used for crowd anomaly detection (CAD). Existing solutions, however, face many challenges, such as low accuracy and high latency due to drones' dynamic shooting distances and angles as well as limited computing and networking capabilities. In this paper, we propose Air-CAD, an edge-assisted multi-drone network that uses air-ground cooperation to achieve fast and accurate CAD. Air-CAD consists of two stages: person detection and multi-feature analysis. To improve CAD accuracy, Air-CAD dynamically adjusts the inference of person detection model based on drones' shooting distances and assigns appropriate feature analysis tasks to drones shooting at variable angles. To achieve fast CAD, edge devices connected to drones are deployed to offload assigned feature analysis tasks from drones. Air-CAD schedules the connection between each drone and edge to accelerate processing based on drone's assigned task and the computing/network resources of the edge device. To validate the performance of Air-CAD, we generate a new simulated human stampede dataset captured from various drone-view recordings. We deploy and evaluate Air-CAD in both simulation and real-world testbed. Experimental results show that Air-CAD achieves 95.33% AUROC and real-time inference latency within 0.47 seconds.
Supplemental Material
MP4 File
presentation video
- Download
- 1161.18 MB
MP4 File
Supplemental video
- Download
- 52.96 MB
References
[1]
2018. Miarmy: Better Crowd Simulation for Maya. http://www.basefount.com/miarmy.html.
[2]
Andra Acsintoae, Andrei Florescu, Mariana-Iuliana Georgescu, Tudor Mare, Paul Sumedrea, Radu Tudor Ionescu, Fahad Shahbaz Khan, and Mubarak Shah. 2022. Ubnormal: New benchmark for supervised open-set video anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 20143--20153.
[3]
Samet Akcay, Dick Ameln, Ashwin Vaidya, Barath Lakshmanan, Nilesh Ahuja, and Utku Genc. 2022. Anomalib: A Deep Learning Library for Anomaly Detection. arXiv:2202.08341 [cs.CV]
[4]
Amnah Aldayri and Waleed Albattah. 2022. Taxonomy of Anomaly Detection Techniques in Crowd Scenes. Sensors 22, 16 (2022), 6080.
[5]
Danilo Avola, Luigi Cinque, Angelo Di Mambro, Anxhelo Diko, Alessio Fagioli, Gian Luca Foresti, Marco Raoul Marini, Alessio Mecca, and Daniele Pannone. 2021. Low-altitude aerial video surveillance via one-class SVM anomaly detection from textural features in UAV images. Information 13, 1 (2021), 2.
[6]
Changrui Chen, Yu Zhang, Qingxuan Lv, Shuo Wei, Xiaorui Wang, Xin Sun, and Junyu Dong. 2019. RRNet: A Hybrid Detector for Object Detection in Drone-Captured Images. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops.
[7]
Kuan-Fu Chen. 2023. It is time to modernize disaster preparedness with crowd analysis. Journal of Acute Medicine 13, 1 (2023), 1.
[8]
Thomas Defard, Aleksandr Setkov, Angelique Loesch, and Romaric Audigier. 2021. Padim: a patch distribution modeling framework for anomaly detection and localization. In International Conference on Pattern Recognition. Springer, 475--489.
[9]
Sutao Deng, Shuai Li, Ke Xie,Wenfeng Song, Xiao Liao, Aimin Hao, and Hong Qin. 2021. A Global-Local Self-Adaptive Network for Drone-View Object Detection. IEEE Transactions on Image Processing 30 (2021), 1556--1569. https://doi.org/10.1109/TIP.2020.3045636
[10]
Naqqash Dilshad, JaeYoung Hwang, JaeSeung Song, and NakMyoung Sung. 2020. Applications and Challenges in Video Surveillance via Drone: A Brief Survey. In 2020 International Conference on Information and Communication Technology Convergence (ICTC). 728--732. https://doi.org/10.1109/ICTC49870.2020.9289536
[11]
Yong-He Dong, Fei Liu, Yong-Mei Liu, Xia-Rong Jiang, and Zhong-Xin Zhao. 2017. Emergency preparedness for mass gatherings: Lessons of" 12.31" stampede in Shanghai Bund. Chinese Journal of Traumatology 20, 04 (2017), 240--242.
[12]
Camille Dupont, Luis Tobias, and Bertrand Luvison. 2017. Crowd-11: A dataset for fine grained crowd behaviour analysis. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 9--16.
[13]
Funda Durup?nar, U?ur Güdükbay, Aytek Aman, and Norman I Badler. 2015. Psychological parameters for crowd simulation: From audiences to mobs. IEEE transactions on visualization and computer graphics 22, 9 (2015), 2145--2159.
[14]
Hao-Shu Fang, Shuqin Xie, Yu-Wing Tai, and Cewu Lu. 2017. Rmpe: Regional multi-person pose estimation. In Proceedings of the IEEE international conference on computer vision. 2334--2343.
[15]
Stephen J Guy, Sujeong Kim, Ming C Lin, and Dinesh Manocha. 2011. Simulating heterogeneous crowd behaviors using personality trait theory. In Proceedings of the 2011 ACM SIGGRAPH/Eurographics symposium on computer animation. 43--52.
[16]
Jordan Henrio and Tomoharu Nakashima. 2018. Anomaly Detection in Videos Recorded by Drones in a Surveillance Context. In 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC). 2503--2508. https://doi.org/10.1109/SMC.2018.00429
[17]
Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7132--7141.
[18]
Eddy Ilg, Nikolaus Mayer, Tonmoy Saikia, Margret Keuper, Alexey Dosovitskiy, and Thomas Brox. 2017. Flownet 2.0: Evolution of optical flow estimation with deep networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2462--2470.
[19]
Cewu Lu, Jianping Shi, and Jiaya Jia. 2013. Abnormal event detection at 150 fps in matlab. In Proceedings of the IEEE international conference on computer vision. 2720--2727.
[20]
Weixin Luo, Wen Liu, and Shenghua Gao. 2017. A revisit of sparse coding based anomaly detection in stacked rnn framework. In Proceedings of the IEEE international conference on computer vision. 341--349.
[21]
Vijay Mahadevan,Weixin Li, Viral Bhalodia, and Nuno Vasconcelos. 2010. Anomaly detection in crowded scenes. In IEEE computer society conference on computer vision and pattern recognition. IEEE, 1975--1981.
[22]
Ramin Mehran, Alexis Oyama, and Mubarak Shah. 2009. Abnormal crowd behavior detection using social force model. In 2009 IEEE conference on computer vision and pattern recognition. IEEE, 935--942.
[23]
Junkun Peng, Qing Li, Yuanzheng Tan, Dan Zhao, Zhenhui Yuan, Jinhua Chen, Hanling Wang, and Yong Jiang. 2023. SkyNet: Multi-Drone Cooperation for Real-Time Identification and Localization. In IEEE International Conference on Computer Communications (INFOCOM). 1--10.
[24]
Hamidreza Rabiee, Javad Haddadnia, Hossein Mousavi, Maziyar Kalantarzadeh, Moin Nabi, and Vittorio Murino. 2016. Novel dataset for fine-grained abnormal behavior understanding in crowd. In 2016 13th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). IEEE, 95--101.
[25]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning. PMLR, 8748--8763.
[26]
Tal Reiss and Yedid Hoshen. 2022. Attribute-based Representations for Accurate and Interpretable Video Anomaly Detection. arXiv preprint arXiv:2212.00789 (2022).
[27]
Khosro Rezaee, Mohammad R. Khosravi, and Maryam Saberi Anari. 2022. Deep-Transfer-Learning-Based Abnormal Behavior Recognition Using Internet of Drones for Crowded Scenes. IEEE Internet of Things Magazine 5, 2 (2022), 41--44. https://doi.org/10.1109/IOTM.001.2100138
[28]
Richard S Sutton. 1990. Integrated modeling and control based on reinforcement learning and dynamic programming. Advances in neural information processing systems 3 (1990).
[29]
Surat Teerapittayanon, Bradley McDanel, and Hsiang-Tsung Kung. 2016. Branchynet: Fast inference via early exiting from deep neural networks. In 2016 23rd international conference on pattern recognition (ICPR). IEEE, 2464--2469.
[30]
Guodong Wang, Shumin Han, Errui Ding, and Di Huang. [n. d.]. Student-teacher feature pyramid matching for anomaly detection. arXiv preprint: 2103.04257 ([n. d.]).
[31]
Xin Wang, Fisher Yu, Zi-Yi Dou, Trevor Darrell, and Joseph E Gonzalez. 2018. Skipnet: Learning dynamic routing in convolutional networks. In Proceedings of the European Conference on Computer Vision (ECCV). 409--424.
[32]
B Yogameena and C Nagananthini. 2017. Computer vision based crowd disaster avoidance system: A survey. International journal of disaster risk reduction 22 (2017), 95--129.
[33]
Zhenhui Yuan, Jie Jin, Jing Chen, Lingling Sun, and Gabriel-Miro Muntean. 2017. ComProSe: Shaping Future Public Safety Communities with ProSe-Based UAVs. IEEE Communications Magazine 55, 12 (2017), 165--171. https://doi.org/10.1109/MCOM.2017.1600869
[34]
Mingjian Zhu, Kai Han, Changbin Yu, and Yunhe Wang. 2020. Dynamic feature pyramid networks for object detection. arXiv preprint arXiv:2012.00779 (2020).
[35]
Xingkui Zhu, Shuchang Lyu, Xu Wang, and Qi Zhao. 2021. TPH-YOLOv5: Improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios. In Proceedings of the IEEE/CVF international conference on computer vision. 2778--2788.
Index Terms
- Air-CAD: Edge-Assisted Multi-Drone Network for Real-time Crowd Anomaly Detection
Recommendations
Multi-Featured Anomaly Detection for Mobile Edge Computing Based UAV Delivery Systems
ACSW '23: Proceedings of the 2023 Australasian Computer Science WeekWith the rapid development of communication infrastructure and the widespread of IoT computing facilities such as Mobile Edge Computing (MEC), the application of Unmanned Aerial Vehicles (UAVs) are increasingly growing especially in last-mile deliveries ...
CloudShield: Real-time Anomaly Detection in the Cloud
CODASPY '23: Proceedings of the Thirteenth ACM Conference on Data and Application Security and PrivacyIn cloud computing, it is desirable if suspicious activities can be detected by automatic anomaly detection systems. Although anomaly detection has been investigated in the past, it remains unsolved in cloud computing. Challenges are: characterizing the ...
Enhancing Drone Security Through Multi-Sensor Anomaly Detection and Machine Learning
AbstractUnmanned Aerial Vehicles (UAVs), commonly referred to as drones, have determined numerous applications across industries, ranging from aerial surveillance to package shipping. As drones are used in vital operations, ensuring cyberattacks and ...
Comments
Information & Contributors
Information
Published In
May 2024
4826 pages
ISBN:9798400701719
DOI:10.1145/3589334
- General Chairs:
- Tat-Seng Chua,
- Chong-Wah Ngo,
- Proceedings Chair:
- Roy Ka-Wei Lee,
- Program Chairs:
- Ravi Kumar,
- Hady W. Lauw
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 May 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- the Major Key Project of PCL
- the National Key R\&D Program of China
- the Shenzhen Key Lab of Software Defined Networking
Conference
WWW '24
Sponsor:
Acceptance Rates
Overall Acceptance Rate 1,899 of 8,196 submissions, 23%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 426Total Downloads
- Downloads (Last 12 months)426
- Downloads (Last 6 weeks)57
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in