Skip to main content
SpringerLink
Log in

Human detection and tracking with deep convolutional neural networks under the constrained of noise and occluded scenes

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Human detection and tracking is a key aspect in surveillance system due to its importance in timely identification of person, recognition of human activity and scene analysis. Convolutional neural networks have been widely used approach in detection and tracking related tasks. In this paper, a robust framework is presented for the human detection and tracking in noisy and occluded environments with the aid of data augmentation techniques. In addition, a softmax layer and integrated loss function is used to improve the detection and classification performance of the proposed model. The primary focus is to perform the human detection task in unconstrained environments. The implemented system outperforms the state-of-the-arts methods which can be validated from the experimental results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. An F, Liu Z (2019) Facial expression recognition algorithm based on parameter adaptive initialization of CNN and LSTM. Vis. Comput. 35:1–16. https://doi.org/10.1007/s0037 1-019-01635 -4

    Article  Google Scholar 

  2. Brunetti A, Buongiorno D, Francesco G, Bevilacqua V (2018) Neurocomputing computer vision and deep learning techniques for pedestrian detection and tracking : A survey. Neurocomputing 300:17–33

    Article  Google Scholar 

  3. Bubryur Kim, N Yuvaraj, KR Sri Preethaa,R Santhosh, A Sabari (2020). Enhanced pedestrian detection using optimized deep convolution neural network for smart building surveillance, Soft Computing.https://doi.org/10.1007/s00500-020-04999-1

  4. Chahyati D, Fanany MI, Arymurthy AM (2017) Tracking people by Detection Using CNN features. Procedia Comput Sci 124:167–172

    Article  Google Scholar 

  5. Coltuc, D; Bolon, P (1999). Strict ordering on discrete images and applications. In Proceedings of the IEEE International Conference on Image Processing, Kobe, Japan, 24–28 October 1999; pp. 150–153

  6. Coltuc D, Bolon P, Chassery J-M (2006) Exact histogram specification. IEEE Trans Image Process 15:1143–1152

    Article  Google Scholar 

  7. Tran Thi Dinh, Nguyen Dinh Vinh, Jeon Jae Wook (2018). Robust Pedestrian Detection via a Recursive Convolution Neural Network, SNPD 2018, June 27–29 2018, Busan

  8. X Du, M El-Khamy, J Lee, and L Davis (2017). “Fused dnn: A deep neural network fusion approach to fast and robust pedestrian detection,” in WACV

  9. Dundar A, Jin J, Martini B, Culurciello E (2017) Embedded streaming deep neural networks accelerator with applications. IEEE TransNeural Netw & Learning Syst 28(7):1572–1583

    Article  MathSciNet  Google Scholar 

  10. Farhadi, A and Redmon, J (2016). YOLO9000: better, Faster, Stronger

  11. Felzenszwalb PF, Girshick RB, McAllester D, Ramanan D (2010) Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal Mach Intell 32:1627–1645

    Article  Google Scholar 

  12. Flores Calero MJ, Aldás M, Lázaro J, Gardel A, Onofa N, Quinga B (2019) Pedestrian Detection under partial occlusion by using logic inference, HOG and SVM. IEEE Latin America Transactions 17(09):1552–1559. https://doi.org/10.1109/TLA.2019.8931190

    Article  Google Scholar 

  13. R Girshick, J Donahue, T Darrell, and J Malik (2014). “Rich feature hierarchies for accurate object detection and semantic segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 580–587

  14. Gonzalez, RC; Woods, RE (2010). Digital image processing, 3rd ed.; Prentice Hall: New Jersey

  15. Guo K, Wu S, Xu YF (2017) Face recognition using both visible light image and near-infrared image and a deep network. CAAI Trans Intell Technol 2(1):39–47

    Article  Google Scholar 

  16. Hajizadeh, MA; Ebrahimnezhad, H (2011). Classification of age groups from facial image using histograms of oriented gradients. In Proceedings of the 7th Iranian Conference on Machine Vision and Image Processing, Iran University of Science and Technology (IUST), Tehran, Iran, 16–17 November 2011; pp. 1–5

  17. Huang, C, Lucey, S, Ramanan, D (2017). Learning policies for adaptive tracking with deep feature cascades. In: Computer Vision Foundation

  18. Jen, TC; Hsieh, B; Wang, SJ (2005). Image contrast enhancement based on intensity-pair distribution.In Proceedings of the IEEE International Conference on Image Processing, Genova, Italy, 11–14 September 2005; pp. 1–4

  19. H Jeon, VD Nguyen and JW Jeon (2019). “Pedestrian Detection Based on Deep Learning,” IECON - 45th annual conference of the IEEE industrial electronics society, Lisbon, Portugal, 2019, pp. 144–151, doi: https://doi.org/10.1109/IECON.2019.8927417

  20. Jian M, Lam K, Dong J, Shen L (2015) Visual-patch-attention-aware saliency Detection. IEEE Transactions on Cybernetics 45(8):1575–1586. https://doi.org/10.1109/TCYB.2014.2356200

    Article  Google Scholar 

  21. Jian M, Qi Q, Dong J, Yin Y, Lam K-M (2018) Integrating QDWD with pattern distinctness and local contrast for underwater saliency detection. Journal of Visual Communication and Image Representation, Volume 53:31–41. https://doi.org/10.1016/j.jvcir.2018.03.008

    Article  Google Scholar 

  22. Jian M, Qi Q, Yu H, Dong J, Cui C, Nie X, Zhang H, Yin Y, Lam K-M (2019) The extended marine underwater environment database and baseline evaluations. Applied Soft Computing, Volume 80:425–437. https://doi.org/10.1016/j.asoc.2019.04.025

    Article  Google Scholar 

  23. Karaaba, M; Surinta, O; Schomaker, L; Wiering, MA (2015). Robust face recognition by computing distances from multiple histograms of oriented gradients. In Proceedings of the IEEE Symposium Series on Computational Intelligence, Cape Town International Convention Center, Cape Town, South Africa, 7–10 December 2015; pp. 203–209

  24. Kim S, Kwak S, Ko BC (2019) Fast Pedestrian Detection in surveillance video based on soft target training of shallow random Forest. IEEE Access 7:12415–12426. https://doi.org/10.1109/ACCESS.2019.2892425

    Article  Google Scholar 

  25. Lan X, Ma AJ, Yuen PC, Chellappa R (2015) Joint sparse representation and robust feature-level fusion for multi-cue visual tracking IEEE trans. Image Process 24(12):5826–5841

    Article  MathSciNet  Google Scholar 

  26. Lei, Z, Chu, R, He, R, Liao, S, Li, SZ (2007). Face recognition by discriminant analysis with Gabor tensor representation. In: International Conference on Biometrics, pp. 87–95. Springer, Berlin

  27. Li J, Liang X, Shen S, Xu T, Yan S (2018) Scale-aware fast R-CNN for pedestrian detection. IEEE Trans Multimedia 20(4):985–996

    Google Scholar 

  28. R Lienhart and J Maydt (2002). “An extended set of haar-like features for rapid object detection,” in ICIP

  29. M Lin, Q Chen, and S Yan (2013). Network in network. arXiv preprint arXiv:1312.4400

  30. J Liu, X Gao, N Bao, J Tang and G Wu (2017), “Deep convolutional neural networks for pedestrian detection with skip pooling,” 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, pp. 2056–2063, doi: https://doi.org/10.1109/IJCNN.2017.7966103

  31. Shu-an Liu, Shi Lv, Hailin Zhang, Jun Gong (2019). Pedestrian Detection Algorithm Based on the Improved SSD, The 31th Chinese Control and Decision Conference (2019 CCDC), IEEE

  32. P Luo, X Wang, and X Tang (2013). Pedestrian parsing via deep Decompositional neural network, in proceedings of IEEE international conference on computer vision (ICCV)

  33. Lv JJ, Cheng C, Tian GD, Zhou XD, Zhou X (2016) Landmark perturbation-based data augmentation for unconstrained face recognition. Signal Process Image Commun 47:465–475

    Article  Google Scholar 

  34. Madbouly AMM, Mostafa M-SM, Wafy M (2015) Performance assessment of feature detector-descriptor combination. Int J ComputSciIssues 12(5):87–94

    Google Scholar 

  35. Mateus A, Ribeiro D, Miraldo P, Nascimento JC (2019) Efficient and robust Pedestrian Detection using deep learning for human-aware navigation. Robot Auton Syst 113:23–37

    Article  Google Scholar 

  36. Pawłowski, P, Piniarski, K (2015). D ˛abrowski, A. Pedestrian detection in low resolution night vision images. In Proceedings of the IEEE Signal Processing: Algorithms, Architectures, Arrangements, and Applications, Pozna ´n, Poland, 23–25 September 2015; pp. 185–190

  37. J Redmon, S Divvala, R Girshick, and A Farhadi (2016). “You only look once: unified, real-time object detection,” in CVPR

  38. KN Renu Chebrolu and PN Kumar (2019). “Deep learning based Pedestrian Detection at all light conditions,” 2019 International Conference on Communication and Signal Processing (ICCSP), Chennai, pp. 0838–0842, doi: https://doi.org/10.1109/ICCSP.2019.8698101

  39. D Ribeiro, A Mateus, JC Nascimento, and P Miraldo (2016). “A real-time pedestrian detector using deep learning for human-aware navigation,“arXiv:1607.04441

  40. Rivera AR, Ryu B, Chae O (2012) Content-aware dark image enhancement through channel division. IEEE Trans Image Process 21:3967–3980

    Article  MathSciNet  Google Scholar 

  41. Yahia Fahem Said, Mohammad Barr (2019). Pedestrian Detection for Advanced Driver Assistance Systems using Deep Learning Algorithms, IJCSNS International Journal of Computer Science and Network Security, VOL.19 No.9, September 2019

  42. Anandamurugan Selvaraj, Jeeva Selvaraj ,Sivabalakrishnan Maruthaiappan, Gokulnath Chandra Babu, Priyan Malarvizhi Kumar (2020). L1 norm based pedestrian detection using video analytics technique, An international journal of computational intelligence, 22 February 2020. https://doi.org/10.1111/coin.12292

  43. Supreeth HSG, Patil CM (2018) Efficient multiple moving object detection and tracking using combined background subtraction and clustering. Signal Image Video Process 15:1097

    Article  Google Scholar 

  44. Y Tian, P Luo, X Wang, and X Tang (2015). “Deep learning strong parts for pedestrian detection,” in ICCV

  45. Wentong Wang , Lichun Wang , Xufei Ge, Jinghua Li and Baocai Yin (2020), Pedestrian Detection Based on Two-Stream UDN, Appl. Sci. , 10, 1866; doi:https://doi.org/10.3390/app10051866

  46. Wang, X, Wang, K, Lian, S (2019). A survey on face data augmentation. arXiv :1904.11685

  47. Wojek C, Dollar P, Schiele B, Perona P (2012) Pedestrian detection:An evaluation of the state of the art. IEEE Trans. Pattern Anal. Mach.Intell. 34(4):743

    Article  Google Scholar 

  48. Xinxin S, Liangnian J, Qinghua L (2019) Detection of stationary humans using time-division UWB MIMO through-wall radar. The Journal of Engineering 2019(20):6799–6802. https://doi.org/10.1049/joe.2019.0542

    Article  Google Scholar 

  49. Chenchen Xu, Guili Wang, Songsong Yan,Jianghua Yu, Baojun Zhang, Shu Dai, Yu Li, and Lin Xu, Fast Vehicle and Pedestrian Detection Using Improved Mask R-CNN (2020). Hindawi, Mathematical Problems in Engineering,Volume 2020, Article ID 5761414, 15 pages. https://doi.org/10.1155/2020/5761414

  50. S Yang, P Luo, CC Loy, and X Tang (2015). “From facial parts responses to face detection: A deep learning approach,” in ICCV

  51. Z Yang and R Nevatia (2016). “A multi-scale cascade fully convolutional network face detector,” in ICPR

  52. Zhang, L Lin, X Liang, K He (2016). Is faster r-cnn doing well for pedestrian detection? In: European Conf. Computer Vision (ECCV), pp. 443–457.

  53. Zhang K, Zhang Z, Li Z, Qiao Y (2016) Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process Lett 23(10):1499–1503

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ATR Key Laboratory, School of Electronics and Information Engineering, Shenzhen University, Shenzhen, China

    Ejaz Ul Haq, Huang Jianjun & Kang Li

  2. Fujian Normal University, Fuzhou, China

    Hafeez Ul Haq

Authors
  1. Ejaz Ul Haq
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huang Jianjun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hafeez Ul Haq
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huang Jianjun.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Haq, E.U., Jianjun, H., Li, K. et al. Human detection and tracking with deep convolutional neural networks under the constrained of noise and occluded scenes. Multimed Tools Appl 79, 30685–30708 (2020). https://doi.org/10.1007/s11042-020-09579-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09579-x

Keywords

Search

Navigation