Skip to main content
SpringerLink
Log in

Real time object detection and trackingsystem for video surveillance system

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper introduces a system capable of real-time video surveillance in low-end edge computing environment by combining object detection tracking algorithm. Recently, the accuracy of object detection has been improved due to the performance of approaches based on deep learning algorithm such as region-based convolutional network, which has two stages for inferencing. One-stage detection algorithms such as single shot detector and you only look once (YOLO) have been developed at the expense of some accuracy and can be used for real-time systems. However, high-performance hardware such as general-purpose graphics processing unit is required to achieve excellent object detection performance and speed. In this study, we propose an approach called N-YOLO which is instead of resizing image step in YOLO algorithm, it divides into fixed size images used in YOLO and merges detection results of each divided sub-image with inference results at different times using correlation-based tracking algorithm the amount of computation for object detection and tracking can be significantly reduced. In addition, we propose a system that can guarantee real-time performance in various edge computing environments by adaptively controlling the cycle of object detection and tracking.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3.
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Al-masni MA, Al-antari MA, Park J-M et al (2018) Simultaneous detection and classification of breast masses in digital mammograms via a deep learning YOLO-based CAD system. Comput Methods Prog Biomed 157:85–94. https://doi.org/10.1016/j.cmpb.2018.01.017

    Article  Google Scholar 

  2. Cao X, Guo S, Lin J, Zhang W, Liao M (2020) Online tracking of ants based on deep association metrics: method, dataset and evaluation. Pattern Recogn 103:107233. https://doi.org/10.1016/j.patcog.2020.107233

    Article  Google Scholar 

  3. Danelljan M, Häger G, Khan FS, Felsberg M (2014) Accurate scale estimation for robust visual tracking. In: BMVC 2014 - proceedings of the British machine vision conference 2014. British Machine Vision Association, BMVA, pp. 1–11

  4. Ding Z, Wong E (2019) Confidence trigger detection: an approach to build real-time tracking-by-detection system. ArXiv 1902(00615):1–9

    Google Scholar 

  5. Durand T Tracking things in object detection videos. In: Move Lab https://www.move-lab.com/blog/tracking-things-in-object-detection-videos. Accessed 27 Jun 2020

  6. Fu H, Wu L, JianM YY, Wang X (2019) MF-SORT: simple online and Realtime tracking with motion features. In: Zhao Y, Barnes N, Chen B et al (eds) Image and graphics. Springer International Publishing, Cham, pp 157–168

    Chapter  Google Scholar 

  7. Geng Y, Liang R-Z, Li W et al (2017) Learning convolutional neural network to maximize Pos@top performance measure. Bruges, Belgium, pp 589–594

    Google Scholar 

  8. Girshick R (2015) Fast R-CNN. In: The IEEE international conference on computer vision (ICCV). IEEE Computer Society, Boston, pp. 1440–1448

  9. Huang C-H, Boyer E, Do B et al (2015) Toward user-specific tracking by detection of human shapes in multi-cameras. In: The IEEE conference on computer vision and pattern recognition (CVPR). IEEE Computer Society, Boston, pp 4027–4035

    Google Scholar 

  10. Le N, Heili A, Odobez JM (2016) Long-term time-sensitive costs for CRF-based tracking by detection. In: lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics). Springer Verlag, pp 43–51

  11. Lin TY, Maire M, Belongie S et al (2014) Microsoft COCO: common objects in context. In: lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics). Springer Verlag, pp 740–755

  12. Nikouei SY, Chen Y, Song S, et al (2018) Real-time human detection as an edge service enabled by a lightweight CNN. In: proceedings - 2018 IEEE international conference on EDGE computing, EDGE 2018 - part of the 2018 IEEE world congress on services. Institute of Electrical and Electronics Engineers Inc., pp 125–129

  13. Pinho RR, Tavares JMR, Correia MV (2007) An improved management model for tracking missing features in computer vision long image sequences. WSEAS Trans Inf Sci Appl 1:196–203

  14. Pinho RR, Tavares JMRS (2009) Tracking features in image sequences with Kalman filtering, global optimization, mahalanobis distance and a management model. Computer Modeling in Engineering & Sciences 46:51–75

  15. Pinho RR, Tavares JMR (2005) Correia MV (2005) a movement tracking management model with Kalman filtering, global optimization techniques and mahalanobis distance. Advances in Computational Methods in Sciences and Engineering 1:1–4

  16. Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger. In: The IEEE conference on computer vision and pattern recognition (CVPR). IEEE Computer Society, Hawaii, pp 7263–7271

  17. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: The IEEE conference on computer vision and pattern recognition (CVPR). IEEE Computer Society, Las Vegas, pp. 779–788

  18. Tavares J, Padilha A (1995)Matching lines in image sequences with geometric constraints. Portugal, pp 1–3

  19. Wang S, Niu L, Li N (2018) Research on image recognition of insulators based on YOLO algorithm. In: 2018 international conference on power system technology (POWERCON). Pp 3871–3874

  20. Yuan Y, Xiong Z, Wang Q (2017) An incremental framework for video-based traffic sign detection, tracking, and recognition. IEEE Trans Intell Transp Syst 18:1918–1929. https://doi.org/10.1109/TITS.2016.2614548

    Article  Google Scholar 

  21. Zhang G, Liang G, Su F, Qu F, Wang JY (2018) Cross-domain attribute representation based on convolutional neural network. In: Huang D-S, GromihaMM, Han K, Hussain A (eds) Intelligent computing methodologies. Springer International Publishing, Cham, pp 134–142

  22. Zhang B, Huang Z, Rahi BH, Wang Q, Li M (2019) Online semi-supervised multi-person tracking with gaussian process regression. MATEC Web Conf 277:01003. https://doi.org/10.1051/matecconf/201927701003

  23. Zhang G, Liang G, LiW FJ, Wang J, Geng Y, Wang JY (2017) Learning convolutional ranking-score function by query preference regularization. In: Yin H, Gao Y, Chen S et al (eds) Intelligent data engineering and automated learning – IDEAL 2017. Springer International Publishing, Cham, pp 1–8

  24. Zhu J, Zhang S, Yang J (2019) Online multi-object tracking using single object tracker and Markov clustering. In: Zhao Y, Barnes N, Chen B et al (eds) Image and graphics. Springer International Publishing, Cham, pp 555–567

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Lovely Professional University, Phagwara, Punjab, 144411, India

    Sudan Jha

  2. Department of Convergence Science, Kongju National University, Gongju, 32588, South Korea

    Changho Seo

  3. Department of Financial Information Security, Kookmin University, Seoul, 02707, South Korea

    Eunmok Yang

  4. Department of Computer Science and Engineering, Sejong University, Seoul, 05006, South Korea

    Gyanendra Prasad Joshi

Authors
  1. Sudan Jha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Changho Seo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eunmok Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gyanendra Prasad Joshi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gyanendra Prasad Joshi.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jha, S., Seo, C., Yang, E. et al. Real time object detection and trackingsystem for video surveillance system. Multimed Tools Appl 80, 3981–3996 (2021). https://doi.org/10.1007/s11042-020-09749-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09749-x

Keywords

Search

Navigation