Skip to main content

Advertisement

SpringerLink
Log in

Recognizing human behaviors from surveillance videos using the SSD algorithm

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The aim is to better recognize human behaviors from surveillance videos. Human behavior recognition technology based on surveillance videos is researched, given the intellectual development of massive surveillance video data with full coverage. This technology builds a human behavior detection and recognition model using the new Single Shot MultiBox Detector (SSD) algorithm, which improves the recognition accuracy. The constructed model’s effectiveness is verified through comparisons with other traditional human behavior recognition algorithms via the TensorFlow framework. Results demonstrate the SSD model-based recognition algorithm’s accuracy is significantly higher than that of Direct Part Marking and Fast Convolutional Neural Network (CNN) algorithms. SSD’s average speed is 0.146 s/frame, and the average accuracy on different datasets is 82.8%. If the target is close or partially occluded, the SSD algorithm can also accurately detect the central target, and the detection efficiency is twice that of the R-CNN algorithm. The algorithm proposed has a simple structure and fast processing speed, which can solve the problems in target detection. The research results can provide a theoretical basis for the research on target detection related to human behavior recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Note: A1 is the Weizmann dataset, A2 is the KTH human movement behavior dataset, A3 is the PETS2004 dataset, and A4 is the Pascal Voc dataset

Similar content being viewed by others

References

  1. Toelstede B (2019) Democracy interrupted: the anti-social side of intensified policing. Democr Secur 15:137–149

    Article  Google Scholar 

  2. Vietz GJ, Walsh CJ, Fletcher TD (2016) Urban hydrogeomorphology and the urban stream syndrome: treating the symptoms and causes of geomorphic change. Prog Phys Geogr 40:480–492

    Article  Google Scholar 

  3. Shi X, Yang C, Xie W, Liang C, Shi Z, Chen J (2018) Anti-drone system with multiple surveillance technologies: architecture, implementation, and challenges. IEEE Commun Mag 56:68–74

    Article  Google Scholar 

  4. Kunz M, Seuss D, Hassan T, Garbas JU, Siebers M, Schmid U, Schöberl M, Lautenbacher S (2017) Problems of video-based pain detection in patients with dementia: a road map to an interdisciplinary solution. BMC Geriatr 17:1–8

    Article  Google Scholar 

  5. Prasad DK, Prasath CK, Rajan D, Rachmawati L, Rajabaly E, Quek C (2016) Challenges in video based object detection in maritime scenario using computer vision, pp 435–440. arXiv preprint arXiv:1608.01079

  6. Long T, Liang Z, Liu Q (2019) Advanced technology of high-resolution radar: target detection, tracking, imaging, and recognition. Sci China Inf Sci 62:40301–40309

    Article  Google Scholar 

  7. Razakarivony S, Jurie F (2016) Vehicle detection in aerial imagery: a small target detection benchmark. J Vis Commun Image Represent 34:187–203

    Article  Google Scholar 

  8. Dong L, Wang B, Zhao M, Xu W (2017) Robust infrared maritime target detection based on visual attention and spatiotemporal filtering. IEEE Trans Geosci Remote Sens 55:3037–3050

    Article  Google Scholar 

  9. Tsakanikas V, Dagiuklas T (2018) Video surveillance systems-current status and future trends. Comput Electr Eng 70:736–753

    Article  Google Scholar 

  10. Ren Y, Yang J, Zhang Q, Guo Z (2019) Multi-feature fusion with convolutional neural network for ship classification in optical images. Appl Sci 9:4209–4213

    Article  Google Scholar 

  11. Zhang X, Yu Q, Yu H (2018) Physics inspired methods for crowd video surveillance and analysis: a survey. IEEE Access 6:66816–66830

    Article  Google Scholar 

  12. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC. SSD: single shot multibox detector. In: Proceedings of European Conference on Computer Vision; pp 21–37

  13. Wang C, Wang H, Hu B, Wen J, Xu J, Li X (2016) A new spectral-spatial algorithm method for hyperspectral image target detection. Guang pu xue yu guang pu fen xi = Guang pu 36:1163–1169

    Google Scholar 

  14. Zhao Z, Li X, Liu H, Xu C (2020) Improved target detection algorithm based on Libra R-CNN. IEEE Access 8:114044–114056

    Article  Google Scholar 

  15. Fakiris E, Papatheodorou G, Geraga M, Ferentinos G (2016) An automatic target detection algorithm for swath sonar backscatter imagery, using image texture and independent component analysis. Remote Sens 8:373–382

    Article  Google Scholar 

  16. Tannouche A, Sbai K, Rahmoune M, Agounoune R, Rahmani A, Rahmani A (2016) Real time weed detection using a boosted cascade of simple features. Int J Electr Comput Eng (2088–8708) 6:6–14

    Google Scholar 

  17. AbdelRaouf A, Higgins CA, Pridmore T, Khalil MI (2016) Arabic character recognition using a Haar cascade classifier approach (HCC). Pattern Anal Appl 19:411–426

    Article  MathSciNet  Google Scholar 

  18. Wei Y, Tian Q, Guo J, Huang W, Cao J (2019) Multi-vehicle detection algorithm through combining Harr and HOG features. Math Comput Simul 155:130–145

    Article  MathSciNet  MATH  Google Scholar 

  19. Guimarães S, Kenmochi Y, Cousty J, Patrocinio Z, Najman L (2017) Hierarchizing graph-based image segmentation algorithms relying on region dissimilarity: the case of the Felzenszwalb–Huttenlocher method. Math Morphol Theory Appl 2:55–75

    Google Scholar 

  20. Taylor LH, Wallace RM, Balaram D, Lindenmayer JM, Eckery DC, Mutonono-Watkiss B, Parravani E, Nel LH (2017) The role of dog population management in rabies elimination—a review of current approaches and future opportunities. Front Vet Sci 4:109

    Article  Google Scholar 

  21. Khan RU, Zhang X, Kumar R (2019) Analysis of ResNet and GoogleNet models for malware detection. J Comput Virol Hacking Tech 15:29–37

    Article  Google Scholar 

  22. Zou Z, Shi Z (2017) Random access memories: a new paradigm for target detection in high resolution aerial remote sensing images. IEEE Trans Image Process 27:1100–1111

    Article  MathSciNet  MATH  Google Scholar 

  23. Zhang M, Pang K, Gao C, Xin M (2020) Multi-scale aerial target detection based on densely connected inception ResNet. IEEE Access 8:84867–84878

    Article  Google Scholar 

  24. Li S, Dou Y, Niu X, Lv Q, Wang Q (2017) A fast and memory saved GPU acceleration algorithm of convolutional neural networks for target detection. Neurocomputing 230:48–59

    Article  Google Scholar 

  25. Nida N, Irtaza A, Javed A, Yousaf MH, Mahmood MT (2019) Melanoma lesion detection and segmentation using deep region based convolutional neural network and fuzzy C-means clustering. Int J Med Inform 124:37–48

    Article  Google Scholar 

  26. Liu B, Luo J, Huang H (2020) Toward automatic quantification of knee osteoarthritis severity using improved faster R-CNN. Int J Comput Assist Radiol Surg 15:457–466

    Article  Google Scholar 

  27. Li L, Yang Z, Jiao L, Liu F, Liu X (2019) High-resolution SAR change detection based on ROI and SPP net. IEEE Access 7:177009–177022

    Article  Google Scholar 

  28. Gong G-C, Tsai A-Y (2019) Reduced daytime net growth rate of Synechococcus spp. in the East China sea in summer estimated using a dilution approach. Estuar Coast Shelf Sci 219:90–96

    Article  Google Scholar 

  29. Dong R, Xu D, Zhao J, Jiao L, An J (2019) Sig-NMS-based faster R-CNN combining transfer learning for small target detection in VHR optical remote sensing imagery. IEEE Trans Geosci Remote Sens 57:8534–8545

    Article  Google Scholar 

  30. Lei X, Sui Z (2019) Intelligent fault detection of high voltage line based on the faster R-CNN. Measurement 138:379–385

    Article  Google Scholar 

  31. Qi L, Li B, Chen L, Wang W, Dong L, Jia X, Huang J, Ge C, Xue G, Wang D (2019) Ship target detection algorithm based on improved faster R-CNN. Electronics 8:959–973

    Article  Google Scholar 

  32. Wan S, Goudos S (2020) Faster R-CNN for multi-class fruit detection using a robotic vision system. Comput Netw 168:107036

    Article  Google Scholar 

  33. Han J, Liao Y, Zhang J, Wang S, Li S (2018) Target fusion detection of LiDAR and camera based on the improved YOLO algorithm. Mathematics 6:213–225

    Article  Google Scholar 

  34. Wu Z, Chen X, Gao Y, Li Y (2018) Rapid target detection in high resolution remote sensing images using Yolo model. ISPAR 42:1915–1920

    Google Scholar 

  35. Wang Z, Du L, Mao J, Liu B, Yang D (2018) SAR target detection based on SSD with data augmentation and transfer learning. IEEE Geosci Remote Sens Lett 16:150–154

    Article  Google Scholar 

  36. Chen H, Zhang L, Ma J, Zhang J (2019) Target heat-map network: an end-to-end deep network for target detection in remote sensing images. Neurocomputing 331:375–387

    Article  Google Scholar 

  37. Falisse A, Van Rossom S, Gijsbers J, Steenbrink F, van Basten BJ, Jonkers I, van den Bogert AJ, De Groote F (2018) OpenSim versus human body model: a comparison study for the lower limbs during gait. J Appl Biomech 34:496–502

    Article  Google Scholar 

  38. Jang S, Vitale JM, Jyung RW, Black JB (2017) Direct manipulation is better than passive viewing for learning anatomy in a three-dimensional virtual reality environment. Comput Educ 106:150–165

    Article  Google Scholar 

  39. Fang Y, Eglen RM (2017) Three-dimensional cell cultures in drug discovery and development. SLAS Discov Adv Life Sci R&D 22:456–472

    Article  Google Scholar 

  40. Chen Z, Jiang J, Zhou C, Fu S, Cai Z (2019) SuperBF: superpixel-based bilateral filtering algorithm and its application in feature extraction of hyperspectral images. IEEE Access 7:147796–147807

    Article  Google Scholar 

  41. Zhao B, Gao L, Liao W, Zhang B (2017) A new kernel method for hyperspectral image feature extraction. Geo-Spat Inf Sci 20:309–318

    Article  Google Scholar 

  42. Liu B, Yu X, Zhang P, Yu A, Fu Q, Wei X (2017) Supervised deep feature extraction for hyperspectral image classification. IEEE Trans Geosci Remote Sens 56:1909–1921

    Article  Google Scholar 

  43. Chen H, Li H, Xu Z, Zhao Y, He T (2019) Real‐time action feature extraction via fast PCA‐Flow. Concurrency Comput Pract Experience e5507:5507–5513

    Google Scholar 

  44. Anjum A, Das M, Murthy J, Gudennavar S, Gopal R, Bubbly S (2018) Template-based classification of SDSS-GALEX point sources. J Astrophys Astron 39:61–69

    Article  Google Scholar 

  45. Shen H, Xu M, Guez A, Li A, Ran F (2019) An accurate sleep stages classification method based on state space model. IEEE Access 7:125268–125279

    Article  Google Scholar 

  46. Chen Y, Luo Y, Huang W, Hu D, Zheng R-Q, Cong S-Z, Meng F-K, Yang H, Lin H-J, Sun Y (2017) Machine-learning-based classification of real-time tissue elastography for hepatic fibrosis in patients with chronic hepatitis B. Comput Biol Med 89:18–23

    Article  Google Scholar 

  47. Li W, Liu H, Wang Y, Li Z, Jia Y, Gui G (2019) Deep learning-based classification methods for remote sensing images in urban built-up areas. IEEE Access 7:36274–36284

    Article  Google Scholar 

  48. Deng L, Zhu H, Zhou Q, Li Y (2018) Adaptive top-hat filter based on quantum genetic algorithm for infrared small target detection. Multimed Tools Appl 77:10539–10551

    Article  Google Scholar 

  49. Körez A, Barışçı N (2020) Object detection with low capacity GPU systems using improved faster R-CNN. Appl Sci 10:83–92

    Article  Google Scholar 

  50. Fang W, Wang L, Ren P (2019) Tinier-YOLO: a real-time object detection method for constrained environments. IEEE Access 8:1935–1944

    Article  Google Scholar 

  51. Wang H, Yu Y, Cai Y, Chen X, Chen L, Liu Q (2019) A comparative study of state-of-the-art deep learning algorithms for vehicle detection. IEEE Intell Transp Syst Mag 11:82–95

    Article  Google Scholar 

  52. Wang D, Tang J, Zhu W, Li H, Xin J, He D (2018) Dairy goat detection based on faster R-CNN from surveillance video. Comput Electron Agric 154:443–449

    Article  Google Scholar 

  53. Shen L, Shi J, Dong Y, Ying S, Peng Y, Chen L, Zhang Q, An H, Zhang Y (2019) An improved deep polynomial network algorithm for transcranial sonography-based diagnosis of Parkinson’s disease. Cogn Comput 12(3):553–562

    Article  Google Scholar 

  54. Shi J, Zheng X, Li Y, Zhang Q, Ying S (2017) Multimodal neuroimaging feature learning with multimodal stacked deep polynomial networks for diagnosis of Alzheimer’s disease. IEEE J Biomed Health Inform 22:173–183

    Article  Google Scholar 

  55. Matthews AGG, Van Der Wilk M, Nickson T, Fujii K, Boukouvalas A, León-Villagrá P, Ghahramani Z, Hensman J (2017) GPflow: a Gaussian process library using TensorFlow. J Mach Learn Res 18:1299–1304

    MathSciNet  MATH  Google Scholar 

  56. Brandon N, Price PS (2020) Calibrating an agent-based model of longitudinal human activity patterns using the consolidated human activity database. J Eposure Sci Environ Epidemiol 30:194–204

    Article  Google Scholar 

  57. Mazoyer B, Mellet E, Perchey G, Zago L, Crivello F, Jobard G, Delcroix N, Vigneau M, Leroux G, Petit L (2016) BIL&GIN: a neuroimaging, cognitive, behavioral, and genetic database for the study of human brain lateralization. Neuroimage 124:1225–1231

    Article  Google Scholar 

  58. Khan RA, Crenn A, Meyer A, Bouakaz S (2019) A novel database of children’s spontaneous facial expressions (LIRIS-CSE). Image Vis Comput 83:61–69

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Academy of Arts and Design, Tsinghua University, Beijing, China

    Husheng Pan

  2. Graduate School of Technology Management, Kyung Hee University, Yongin-Si, 17104, Korea

    Yuzhen Li

  3. Ticketing Center, Shenzhen Metro Group Co., Ltd, Shenzhen, 518026, China

    Dezhu Zhao

Authors
  1. Husheng Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuzhen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dezhu Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dezhu Zhao.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pan, H., Li, Y. & Zhao, D. Recognizing human behaviors from surveillance videos using the SSD algorithm. J Supercomput 77, 6852–6870 (2021). https://doi.org/10.1007/s11227-020-03578-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-020-03578-3

Keywords

Search

Navigation