Concealed pistol detection from thermal images with deep neural networks

Abstract

Violence involving firearms is a rising threat that requires precise and competent surveillance systems. Current surveillance technologies involve continuous human observation and are prone to human errors. To handle such errors and monitor with minimal human effort, new solutions using artificial intelligence approaches that can detect and pinpoint the threat are required. In this study, our aim is to develop a deep learning-based solution capable of detecting and locating concealed pistols on thermal images for real-time surveillance. For this purpose, we generate a dataset consisting of thermal video recordings of multiple human models and combine this dataset with thermal images from public sources. Then, we build up a deep learning-based framework by combining two deep learning models that detects and localizes the concealed pistol in the given thermal image. We evaluate multiple deep learning architectures for the classification and segmentation of the images. The best test set results in detecting the concealed pistol was achieved by a fine-tuned VGG19-based convolutional neural network model with an F1 score of 0.84 on the test set. In the second module of the system, a fine-tuned Yolo-V3 model trained as a multi-tasking model for both classification and location detection gave the highest mean average precision value of 0.95 in labeling and locating the pistol in a bounding box in approximately 10 milliseconds. The findings exhibit the potential of using deep learning techniques with thermal imaging for the real time concealed pistol detection.