Skip to main content

Advertisement

SpringerLink
Log in

Enhanced pedestrian detection using optimized deep convolution neural network for smart building surveillance

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Pedestrian detection and tracking is a critical task in the area of smart building surveillance. Due to advancements in sensors, the architects concentrate in construction of smart buildings. Pedestrian detection in smart building is greatly challenged by the image noises by various external environmental parameters. Traditional filter-based techniques for image classification like histogram of oriented gradients filters and machine learning algorithms suffer to perform well for huge volume of pedestrian input images. The advancements in deep learning algorithms perform exponentially good in handling the huge volume of image data. The current study proposes a pedestrian detection model based on deep convolution neural network (CNN) for classification of pedestrians from the input images. Proposed optimized version of VGG-16 architecture is evaluated for pedestrian detection on the INRIA benchmarking dataset consisting of 227 × 227 pixel images. The proposed model achieves an accuracy of 98.5%. It was found that proposed model performs better than the other pretrained CNN architectures and other machine learning models. Pedestrians are reasonably detected and the performance of the proposed algorithm is validated.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Data availability

We used our own data.

References

  • Angelova A, Krizhevsky A, Vanhoucke V, Ogale A, Ferguson D (2015) Real-time pedestrian detection with deep network cascades

  • Bartlett MS, Littlewort G, Fasel I, Movellan JR (2003) Real time face detection and facial expression recognition: development and applications to human computer interaction. In: Proceedings of the conference on computer vision and pattern recognition workshop (CVPRW), IEEE, vol 5, p 53

  • Bayoumi A, Karkowski P, Bennewitz M (2019) Speeding up person finding using hidden Markov models. Robot Auton Syst 115:40–48

    Article  Google Scholar 

  • Becherer N, Pecarina J, Nykl S, Hopkinson K (2019) Improving optimization of convolutional neural networks through parameter fine-tuning. Neural Comput Appl 31:3469–3479

    Article  Google Scholar 

  • Beck LF, Dellinger AM, O’Neil ME (2007) Motor vehicle crash injury rates by mode of travel, United States: using exposure-based methods to quantify differences. Am J Epidemiol 166(2):212–218

    Article  Google Scholar 

  • Benenson R, Mathias M, Timofte R, Van Gool L (2012) Pedestrian detection at 100 frames per second. In: Proceedings IEEE conference CVPR, pp 2903–2910

  • Benenson R, Omran M, Hosang J, Schiele B (2015) Ten years of pedestrian detection, what have we learned?. In: Agapito L, Bronstein M, Rother C (eds) Computer vision—ECCV 2014 workshops. Lecture notes in computer science, vol 8926. Springer, Cham

  • Bishop CM (2006) Pattern recognition and machine learning. Springer, New York

    MATH  Google Scholar 

  • Boudjit K, Larbes C (2015) Detection and implementation autonomous target tracking with a quadrotor drone. In: Proceedings of the 12th international conference on informatics in control, automation and robotics (ICINCO), IEEE, pp 223–230

  • Brunetti A, Buongiorno D, Trotta GF, Bevilacqua V (2018) Computer vision and deep learning techniques for pedestrian detection and tracking: a survey. Neurocomputing 300:17–33

    Article  Google Scholar 

  • Cai Z, Fan Q, Feris RS, Vasconcelos N (2016) A unified multi-scale deep convolutional neural network for fast object detection. arXiv:1607.07155v1[cs.CV]

  • Coelingh E, Eidehall A, Bengtsson M (2010) Collision warning with full auto brake and pedestrian detection—a practical example of automatic emergency braking. In: 13th international IEEE conference on intelligent transportation systems (ITSC)

  • Cong J, Xiao B (2014) Minimizing computation in convolutional neural networks. Artificial neural networks and machine learning. Springer, ICANN, pp 281–290

    Google Scholar 

  • Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  • Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Computer vision and pattern recognition, IEEE computer society conference, vol 1, pp 886–893

  • Dinakaran RK, Easom P, Bouridane A, Zhang L, Jiang R, Mehboob F, Rauf A (2020) Deep learning based pedestrian detection at distance in smart cities. In: Bi Y, Bhatia R, Kapoor S (eds) Intelligent systems and applications. Springer, Cham

    Google Scholar 

  • Dollar P, Wojek C, Schiele B, Perona P (2012) Pedestrian detection: an evaluation of the state of the art. IEEE Trans Pattern Anal Mach Intell 34(4):743–761

    Article  Google Scholar 

  • Dollár P, Appel R, Belongie S, Perona P (2014) Fast feature pyramids for object detection. IEEE Trans Pattern Anal Mach Intell 36:1532–1545

    Article  Google Scholar 

  • Dung CV, Anh LD (2019) Autonomous concrete crack detection using deep fully convolutional neural network. Autom Constr Elsevier 99:52–58

    Article  Google Scholar 

  • Enzweiler M, Gavrila DM (2009) Monocular pedestrian detection: survey and experiments. IEEE Trans Pattern Anal Mach Intell 31(12):2179–2195

    Article  Google Scholar 

  • Ess A, Leibe B, Schindler K, van Gool L (2008) A mobile vision system for robust multi-person tracking. In: CVPR

  • Felzenszwalb P, Huttenlocher D (2000) Efficient matching of pictorial structures. In: CVPR, Hilton Head Island, South Carolina, USA, pp 66–75

  • Felzenszwalb PF, Girshick RB, McAllester D, Ramanan D (2010) Object detection with discriminatively trained part based models. IEEE Trans Pattern Anal Mach Intell 32:1627–1645

    Article  Google Scholar 

  • Gall J, Lempitsky V (2013) Class-specific hough forests for object detection. In: Criminisi A, Shotton J (eds) Decision forests for computer vision and medical image analysis. Springer, London, pp 143–157

    Chapter  Google Scholar 

  • Gavrila DM (1999) The visual analysis of human movement: a survey. CVIU 73(1):82–98

    MATH  Google Scholar 

  • Geronimo D, Lopez AM, Sappa AD, Graf T (2010) Survey of pedestrian detection for advanced driver assistance systems. IEEE Trans Pattern Anal Mach Intell 32(7):1239–1258

    Article  Google Scholar 

  • He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: CVPR

  • Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR (2012) Improving neural networks by preventing co-adaptation of feature detectors. arXiv:1207.0580

  • Hou Y-L, Song Y, Hao X, Shen Y, Qian M (2017) Multispectral pedestrian detection based on deep convolutional neural networks. In: Proceedings of IEEE international conference on signal processing, communications and computing, pp 1–4

  • Huang K, Wang L, Tan T, Maybank S (2008) A real-time object detecting and tracking system for outdoor night surveillance. Pattern Recognit 41:432–444

    Article  Google Scholar 

  • Huang J, Rathod V, Sun C, Zhu M, Korattikara A, Fathi A et al (2017) Speed/accuracy trade-offs for modern convolutional object detectors. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 3296–3305

  • Ioffe S, Forsyth DA (2001) Probabilistic methods for finding people. IJCV 43(1):45–68

    Article  Google Scholar 

  • Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM international conference on multimedia, ACM, pp 675–678

  • Kim JH, Hong HG, Park KR (2017) Convolutional neural network-based human detection in nighttime images using visible light camera sensors. Sensors 17:1–26

    Article  Google Scholar 

  • Kim JH, Batchuluun G, Park KR (2018) Pedestrian detection based on faster R-CNN in nighttime by fusing, deep convolutional features of successive images. Expert Syst Appl 114:15–33

    Article  Google Scholar 

  • Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Proceedings of the advances in neural information processing systems, pp 1097–1105

  • Lai CQ, Teoh SS (2014) A review on pedestrian detection techniques based on histogram of oriented gradient feature. In: Proceedings of IEEE conference on research and development (SCOReD), pp 1–6

  • Lavin A, Gray S (2015) Fast algorithms for convolutional neural networks. arXiv:1509.09308v2[cs.NE]

  • Levinson J, Askeland J, Becker J, Dolson J, Held D, Kammel S, Kolter JZ, Langer D, Pink O, Pratt V et al (2011) Towards fully autonomous driving: systems and algorithms. In: Proceedings of the intelligent vehicles symposium (IV), IEEE, pp 163–168

  • Li H, Wu Z, Zhang J (2016) Pedestrian detection based on deep learning model. In: 9th international congress on image and signal processing, biomedical engineering and informatics, IEEE, pp 796–800

  • Li J, Liang X, Shen S, Xu T, Feng J, Yan S (2018) Scale-aware fast R-CNN for pedestrian detection. IEEE Trans Multimed 20:985–996

    Google Scholar 

  • Liu D, Wang X, Song J (2015) A robust pedestrian detection based on corner tracking. In: International conference on information, science and technology, pp 207–211

  • Liu W, Liao S, Hasan I (2019) Center and scale prediction: a box-free approach for object detection. arXiv:1904.02948v2[cs.CV]

  • Llorca DF, Parra I, Quintero R, Fernández C, Izquierdo R, Sotelo M (2014) Stereo-based pedestrian detection in crosswalks for pedestrian behavioural modelling assessment. In: International conference information in control, automation and robotics, pp 102–109

  • Majaranta P, Bulling A (2014) Eye tracking and eye-based human–computer interaction. In: Proceedings of the advances in physiological computing, Springer, pp 39–65

  • Mateus A, Ribeiro D, Miraldo P, Nascimento JC (2018) Efficient and robust pedestrian detection using deep learning for human-aware navigation. Robot Auton Syst. arXiv:1607.04441v3[cs.RO]

  • Mateus A, Ribeiro D, Miraldo P, Nascimento JC (2019) Efficient and robust pedestrian detection using deep learning for human-aware navigation. Robot Auton Syst 113:23–37

    Article  Google Scholar 

  • Mohan A, Papageorgiou C, Poggio T (2001) Detection in images by components. PAMI 23(4):349–361

    Article  Google Scholar 

  • Overett G, Petersson L, Brewer N, Andersson L, Pettersson N (2008) A new pedestrian dataset for supervised learning. In: Proceedings of IEEE intelligent vehicles symposium, pp 373–378

  • Rautaray SS, Agrawal A (2015) Vision based hand gesture recognition for human computer interaction: a survey. Artif Intell Revol 43:1–54

    Article  Google Scholar 

  • Rhodin H, Robertini N, Casas D, Richardt C, Seidel H-P, Theobalt C (2016) General automatic human shape and motion capture using volumetric contour cues. In: Proceedings of the European conference on computer vision, Springer, pp 509–526

  • Ronfard R, Schmid C, Triggs B (2002) Learning to parse pictures of people. In: The 7th ECCV, Copenhagen, Denmark, vol IV, pp 700–714

  • Shen J, Xiong X, Xue Z, Bian Y (2019) A convolutional neural-network-based pedestrian counting model for various crowded scenes. Comput Aided Civ Infrastruct Eng 34:897–914

    Article  Google Scholar 

  • Spencer BF Jr, Hoskere V, Narazaki Y (2019) Advances in computer vision-based civil infrastructure inspection and monitoring. Engineering 5:199–222

    Article  Google Scholar 

  • Tian Y, Luo P, Wang X, Tang X (2015) Deep learning strong parts for pedestrian detection. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 1904–1912

  • Unies N, U.N.E.C. for Europe et al (2015) Statistics of road traffic accidents in Europe and North America. United Nations, New York

    Google Scholar 

  • Urmson C et al (2008) Self-driving cars and the urban challenge. IEEE Intell Syst 23:66–68

    Article  Google Scholar 

  • Viola P, Jones MJ, Snow D (2003) Detecting pedestrians using patterns of motion and appearance. In: The 9th ICCV, Nice, France, vol 1, pp 734–741

  • Wagner J, Fischer V, Herman M, Behnke S (2016) Multispectral pedestrian detection using deep fusion convolutional neural networks. In: Proceedings of European symposium on artificial neural networks, computational intelligence, and machine learning, pp 509–514

  • Wang L, Ouyang W, Wang X, Lu H (2015) Visual tracking with fully convolutional networks. In: Proceedings of the IEEE international conference on computer vision, pp 3119–3127

  • Wang Y, Pierard S, Song-Zhi S, Jodoin P-M (2017) Improving pedestrian detection using motion-guided filtering. Pattern Recognit Lett 96:106–112

    Article  Google Scholar 

  • Wu J, Rehg JM (2011) CENTRIST: a visual descriptor for scene categorization. IEEE Trans Pattern Anal Mach Intell 33:1489–1500

    Article  Google Scholar 

  • Wu Z, Yuan J, Zhang J, Huang H (2016a) A hierarchical face recognition algorithm based on humanoid nonlinear least-squares computation. J Ambient Intell Humaniz Comput 7:229–238

    Article  Google Scholar 

  • Wu Z, Yu Z, Yuan J, Zhang J (2016b) A twice face recognition algorithm. Soft Comput 20(3):1007–1019

    Article  Google Scholar 

  • Xiao T, Li S, Wang B, Lin L, Wang X (2017) Joint detection and identification feature learning for person search. arXiv:1604.01850v3[cs.CV]

  • Zeng X, Ouyang W, Wang X (2013) Multi-stage contextual deep learning for pedestrian detection. In: ICCV

  • Zhang C, Patras P, Haddadi H (2019) Deep learning in mobile and wireless networking: a survey. IEEE Commun Surv Tutor 21:2224–2287

    Article  Google Scholar 

  • Zitnick CL, Dollar P (2014) Edge boxes: locating object proposals from edges. In: ECCV

Download references

Acknowledgements

This work was supported by Korea Research Fellowship Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT (No. 2019H1D3A1A01101442). This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea Government (MSIT) (No. 2019R1G1A1095215).

Author information

Authors and Affiliations

  1. Department of Architectural Engineering, Kyungil University, Gyeongbuk, South Korea

    Bubryur Kim & N. Yuvaraj

  2. Department of CSE, KPR Institute of Engineering and Technology, Coimbatore, India

    K. R. Sri Preethaa

  3. Department of CSE, Faculty of Engineering, Karpagam Academy of Higher Education, Coimbatore, India

    R. Santhosh

  4. Department of IT, K S Rangasamy College of Technology, Tiruchengode, India

    A. Sabari

Authors
  1. Bubryur Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. N. Yuvaraj
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. K. R. Sri Preethaa
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. R. Santhosh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. A. Sabari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to K. R. Sri Preethaa.

Ethics declarations

Conflict of interest

The authors state that there is no conflict of interest.

Human and animal rights statement

Humans and animals are not involved in this research work.

Additional information

Communicated by V. Loia.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, B., Yuvaraj, N., Sri Preethaa, K.R. et al. Enhanced pedestrian detection using optimized deep convolution neural network for smart building surveillance. Soft Comput 24, 17081–17092 (2020). https://doi.org/10.1007/s00500-020-04999-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-020-04999-1

Keywords

Search

Navigation