Skip to main content

Advertisement

SpringerLink
Log in

Real-time moving human detection using HOG and Fourier descriptor based on CUDA implementation

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Real-time applications of image and video processing algorithms have seen explosive growth in number and complexity over the past decade driven by consumer, scientific and defense applications exploiting inexpensive digital video cameras and networked computing device. This growth has opened up different alternatives to greatly enhance the surveillance capabilities using new architectures and parallelization strategies developed due to the increased accessibility of multicore, multi-threaded processors along with general purpose graphics processing units (GPUs). In this paper, we present a new implementation of a moving human detection algorithm on GPU based on the programming language CUDA. In our approach, the moving object is extracted by background subtraction based on the GMM (Gaussian Mixture Model) on GPU. Then, two complementary features are extracted for moving object classification. They are contour-based description: FD or Fourier Descriptor and region-based description: HOG or Histogram of Oriented Gradient. Both descriptors will then be effectively integrated to SVM (Support Vector Machine), which is able to provide the posterior probability, to achieve better performance. The implementation of such algorithm on a GPU allows a great performance in terms of execution time since it is 19 times faster than that on a CPU. Experimental results show also that the proposed approach outperforms some existing techniques and can detect pedestrians in real-time effectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Shashua, A., Gdalyahu, Y., Hayun, G.: Pedestrian detection for driving assistance systems: single-frame classification and system level performance. IEEE Intell. Veh. Symp. IV 2004, 1–6 (2004)

    Google Scholar 

  2. Zhao, L., Thorpe, C.E.: Stereo-and neural network-based pedestrian detection. IEEE Trans. Intell. Transp. Syst. 1(3), 148–154 (2000)

    Article  Google Scholar 

  3. Viola, P., Jones, M.J., Snow, D.: Detecting pedestrians using patterns of motion and appearance. Int. J. Comp. Vis. 63(2), 153–161 (2005)

    Article  Google Scholar 

  4. Papageorgiou, C., Poggio, T.: Trainable pedestrian detection. In: Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348), Kobe, vol. 4, pp. 35–39 (1999). https://doi.org/10.1109/ICIP.1999.819462

  5. Hogg, D.: Model-based vision: a program to see a walking person. Image Vis. Comput. 1(1), 5–20 (1983)

    Article  Google Scholar 

  6. Guo, Y., Xu, G., Tsuji, S.: Understanding human motion patterns. In: Proceedings of the 12th IAPR International Conference on Pattern Recognition, vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5), vol. 2, pp. 325–329 (1994). https://doi.org/10.1109/ICPR.1994.576929

  7. Rohr, K.: Towards model-based recognition of human movements in image sequences. CVGIP Image Underst. 59(1), 94–115 (1994)

    Article  Google Scholar 

  8. Szarvas, M., Yoshizawa, A., Yamamoto, M., Ogata, J.: Pedestrian detection with convolutional neural networks. In: IEEE Proceedings. Intelligent Vehicles Symposium, Las Vegas, NV, USA, pp. 224–229 (2005). https://doi.org/10.1109/IVS.2005.1505106

  9. Zhu, Q., Yeh, M.C., Cheng, K.T., Avidan, S.: Fast human detection using a cascade of histograms of oriented gradients. IEEE Comp. Soc. Conf. Comp. Vis. Pattern Recognit. 2, 1491–1498 (2006)

    Google Scholar 

  10. Banerjee, P., Sengupta, S.: Human motion detection and tracking for video surveillance. In: Proceedings of the National Conference of Communications, IIT Bombay, Mumbai, pp. 88–92 (2008)

  11. Wang, X., Han, T.X., Yan, S.: An HOG-LBP human detector with partial occlusion handling. In: 2009 IEEE 12th International Conference on Computer Vision, Kyoto, pp. 32–39 (2009). https://doi.org/10.1109/ICCV.2009.5459207

  12. Bolme, D.S., Lui, Y.M., Draper, B.A., Beveridge, J.R.: Simple real-time human detection using a single correlation filter. In: 2009 12th IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, Snowbird, UT, pp. 1–8 (2009). https://doi.org/10.1109/PETS-WINTER.2009.5399555

  13. DeCann, B., Ross, A.: Gait curves for human recognition, backpack detection, and silhouette correction in a nighttime environment. In: Proc. SPIE 7667, Biometric Technology for Human Identification VII, 76670Q (2010). https://doi.org/10.1117/12.851296

  14. Barnich, O.: Motion detection and human recognition in video sequences, Thesis report, Faculty of Engineering and Computer Science, University of Liège (2010)

  15. Nguyen, D.T., Li, W., Ogunbona, P.O.: Human detection from images and videos: a survey. Pattern Recognit. 51, 148–175 (2016)

    Article  Google Scholar 

  16. Thanh N.D.: Human detection from images and video. Thesis report, College of Engineering and Computer Science, University of Central Florida—Orlando, FL (2012)

  17. Paul, M., Haque, S.M., Chakraborty, S.: Human detection in surveillance videos and its applications—a review. EURASIP J. Adv. Sig. Process. 2013(1), 176 (2013)

    Article  Google Scholar 

  18. Chesnais T.: Contextualization of a pedestrian detector: Application to the surveillance of public spaces. Thesis report, Blaise Pascal University, Clermont-Ferrand II (2013)

  19. Ouyang, W., Wang, X.: Joint deep learning for pedestrian detection. In: 2013 IEEE International Conference on Computer Vision, Sydney, NSW, pp. 2056–2063 (2013). https://doi.org/10.1109/ICCV.2013.257

  20. Dehghan, A., Idrees, H., Zamir, A.R., Shah, M.: Automatic detection and tracking of pedestrians in videos with various crowd densities. In: Weidmann, U., Kirsch, U., Schreckenberg, M. (eds.) Pedestrian and Evacuation Dynamics 2012, pp. 3–19. Springer, Cham (2014)

    Chapter  Google Scholar 

  21. Bourdev, L.D., Yang, F., Fergus, R.: Deep poselets for human detection. CoRR. arXiv preprint arXiv:1407.0717 (2014)

  22. Mahapatra, A., Mishra, T.K., Sa, P.K., Majhi, B.: Human recognition system for outdoor videos using Hidden Markov model. AEU Int. J. Electron. Commun. 68(3), 227–236 (2014)

    Article  Google Scholar 

  23. Luo, P., Tian, Y., Wang, X., Tang, X.: Switchable deep network for pedestrian detection. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, pp. 899–906 (2014). https://doi.org/10.1109/CVPR.2014.120

  24. Lu, Y., Boukharouba, K., Boonært, J., Fleury, A., Lecoeuche, S.: Application of an incremental SVM algorithm for on-line human recognition from video surveillance using texture and color features. Neurocomput. J. 126, 132–140 (2014)

    Article  Google Scholar 

  25. Wicaksono, I.B., An, F., Mattausch, H.J.: Memory-based hardware-accelerated system for high-speed human detection. Adv. Robot. 28(5), 317–327 (2014)

    Article  Google Scholar 

  26. Emami, A.: Occlusion Handling in Video Surveillance Systems. Thesis report, Faculty of Engineering, Architecture and Information Technology, University of Queensland, (2015)

  27. Hatto, M.: Acceleration of pedestrian detection system using hardware-software co-design. MSc Thesis, Lund University (2015)

  28. Jiang, Y., Ma, J.: Combination features and models for human detection. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, pp. 240–248 (2015). https://doi.org/10.1109/CVPR.2015.7298620

  29. Angelova, A., Krizhevsky, A., Vanhoucke, V., Ogale, A.S., Ferguson, D.: Real-time pedestrian detection with deep network cascades. In: The British Machine Vision Conference, BMVC 2015, vol. 2, pp. 4–16, September (2015)

  30. Ramin, M.: Improvements to tracking pedestrians in video streams using a pre-trained convolutional neural network. Electronic Thesis and Dissertation Repository. 3886 (2016). https://ir.lib.uwo.ca/etd/3886

  31. Ribeiro, D., Mateus, A., Miraldo, P., Nascimento, J.C.: A real-time pedestrian detector using deep learning for human-aware navigation. In: 2017 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), Coimbra, pp. 165–171 (2017). https://doi.org/10.1109/ICARSC.2017.7964070

  32. Suleiman, A., Sze, V.: An energy-efficient hardware implementation of HOG-based object detection at 1080HD 60 fps with multi-scale support. J. Sig. Process. Syst. 84(3), 325–337 (2016)

    Article  Google Scholar 

  33. Campmany, V., Silva, S., Espinosa, A., Moure, J.C., Vázquez, D., López, A.M.: GPU-based pedestrian detection for autonomous driving. Procedia. Comput. Sci. 80, 2377–2381 (2016)

    Article  Google Scholar 

  34. Zhang, M., Xin, M.: Human detection using random color similarity feature and random ferns classifier. PLoS One J. 11(9), e0162830 (2016)

    Article  MathSciNet  Google Scholar 

  35. Lee, N., Weng, X., Boddeti, V.N., Zhang, Y., Beainy, F., Kitani, K., Kanade, T.: Visual compiler: synthesizing a scene-specific pedestrian detector and pose estimator. arXiv preprint arXiv:1612.05234 (2016)

  36. Kim, J.H., Hong, H.G., Park, K.R.: Convolutional neural network-based human detection in nighttime images using visible light camera sensors. Sensors 17(5), 1065 (2017)

    Article  Google Scholar 

  37. AlDahoul, N., Sabri, M., Qalid, A., Mansoor, A.M.: Real-time human detection for aerial captured video sequences via deep models. Comput. Intell. Neurosci. 2018, 14 (2018). https://doi.org/10.1155/2018/1639561

    Article  Google Scholar 

  38. Almonfrey, D., do Carmo, A.P., de Queiroz, F.M., Picoreti, R., Vassallo, R.F., Salles, E.O.T.: A flexible human detection service suitable for Intelligent Spaces based on a multi-camera network. Int. J. Distrib. Sensor Netw. 14(3), 1550147718763550 (2018)

    Article  Google Scholar 

  39. Afifi, M., Ali, Y., Amer, K., Shaker, M., ElHelw, M.: Robust real-time pedestrian detection in aerial imagery on Jetson TX2. arXiv preprint arXiv:1905.06653 (2019)

  40. Vandersteegen, M., Van Beeck, K., Goedemé, T.: Real-time multispectral pedestrian detection with a single-pass deep neural network. International Conference Image Analysis and Recognition, pp. 419–426. Springer, Cham (2018)

    Google Scholar 

  41. Permuter, H., Francos, J., Jermyn, I.H.: Gaussian mixture models of texture and colour for image database retrieval. In: 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings (ICASSP ’03), Hong Kong, pp. III-569 (2003). https://doi.org/10.1109/ICASSP.2003.1199538

  42. Liu, T., Stathaki, T.: Faster R-CNN for robust pedestrian detection using semantic segmentation network. Front Neurorobot 12, 64 (2018)

    Article  Google Scholar 

  43. Mao, J., Xiao, T., Jiang, Y. Cao, Z.: What can help pedestrian detection?. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3127–3136 (2017)

  44. Bahri, H., Chouchene, M., Khemiri, R., Sayadi, F., Atri, M.: March. fast moving human detection using fourier and HOG descriptors based CUDA. In: IEEE 15th International Multi-Conference on Systems, Signals & Devices (SSD), pp. 202–207 (2018)

  45. Flohr, F., Gavrila, D.M.: PedCut: an iterative framework for pedestrian segmentation combining shape models and multiple data cues. In: BMVC (2013)

  46. Jeannin, S., Bober, M.: Description of core experiments for mpeg-7 motion/shape. MPEG-7, ISO/IEC/JTC1/SC29/WG11/MPEG99/N2690, Seoul (1999)

  47. Zivkovic, Z.: Improved adaptive Gaussian mixture model for background subtraction. In: Proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004, vol. 2, pp. 28–31, August (2004)

  48. Zivkovic, Z., Van Der Heijden, F.: Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recognit. Lett. 27(7), 773–780 (2006)

    Article  Google Scholar 

  49. Pham, V., Vo, P., Hung, V.T.: GPU implementation of extended gaussian mixture model for background subtraction. IEEE Int. Conf. Comput. Commun. Technol. Res. Innov. Vis. Future 2010, 1–4 (2010)

    Google Scholar 

  50. Dariu M. Gavrila, PedCut 2013 Segmentation Dataset. Online: http://www.lookingatpeople.com/download-daimler-ped-segm-benchmark/index.html (2013). Accessed 01 May 2019

  51. Richard Ralph, MPEG-7 Core Experiment CE-Shape-1 Test Set. Online: http://www.dabi.temple.edu/~shape/MPEG7/dataset.html (1999). Accessed 01 May 2019

  52. Rui Zhao, CUHK01 Dataset. Online: http://www.ee.cuhk.edu.hk/~rzhao/ (2017). Accessed 01 May 2019

  53. Mary Pat Fitzgerald, Pedestrian Data. Online: http://cbcl.mit.edu/software-datasets/PedestrianData.html (2000). Accessed 01 May 2019

  54. Navneet Dalal, INRIA Person Dataset. Online: http://pascal.inrialpes.fr/data/human/ (2016). Accessed 01 May 2019

  55. Bahri, H., Sayadi, F., Khemiri, R., Chouchene, M., Atri, M.: Image feature extraction algorithm based on CUDA architecture: case study GFD and GCFD. IET Comput. Dig. Techniq. 11(4), 125–132 (2017)

    Article  Google Scholar 

  56. Pedersoli, M.; Gonzàlez i Sabaté, J., dir.; Roca, X.: Hierarchical multiresolution models for fast object detection. [Barcelona]: Universitat Autònoma de Barcelona, 2015. 1 recurs electrònic (139 p.). ISBN 9788449032066. Tesi doctoral, Departament de Ciències de la Computació, Universitat Autònoma de Barcelona (2012) [Checked: 5 decsember 2019]

  57. Smach, F., Miteran, J., Atri, M., et al.: An FPGA-based accelerator for Fourier descriptors computing for color object recognition using SVM. J. Real Time Image Process 2(4), 249–258 (2007)

    Article  Google Scholar 

  58. NVIDIA, C., CUDA Occupancy Calculator. CUDA SDK. Online: https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwjOhIu8L7XAhXQYlAKHWJuA5QQFggmMAA&url=https%3A%2F%2Fdeveloper.download.nvidia.com%2Fcompute%2Fcuda%2FCUDA_Occupancy_calculator.xls&usg=AOvVaw3C1_WHEkOfxeH1sjzxGYB5 (2010). Accessed 01 May 2019

  59. Davis, J.W., Sharma, V.: Background-subtraction using contour-based fusion of thermal and visible imagery. Comput. Vis. Image Underst. 106, 162–182 (2007)

    Article  Google Scholar 

  60. Sudowe, P., Leibe, B.: Efficient Use of Geometric Constraints for Sliding-Window Object Detection in Video. In IEEE 8th International Conference on Computer Vision Systems, ICVS 2011, Sophia, pp. 11–20, September (2011)

  61. Pedersoli, M., Gonzàlez i Sabaté, J., Roca, X.: Hierarchical multiresolution models for fast object detection, University Autònoma of Barcelona. 1 recurs electrònic (139 p.). ISBN 9788449032066. Thesis doctoral report—Universitat Autònoma de Barcelona. Departament de Ciències de la Computació. Online: https://ddd.uab.cat/record/130257(2012). Accessed 01 May 2019

  62. Ahmed Magdi Osman, GPU-HOG. Online: https://github.com/ahmedmagdiosman/GPU-HOG (2015). Accessed 01 May 2019

  63. Fleuret, F., Berclaz, J., Lengagne, R., Fua, P.: Multicamera people tracking with a probabilistic occupancy map. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 267–282 (2008). https://doi.org/10.1109/TPAMI.2007.1174

    Article  Google Scholar 

  64. Alex Leykin, Dataset 03: OSU Color and Thermal Database. Online: http://vcipl-okstate.org/pbvs/bench/Data/03/download.html (2007). Accessed 01 May 2019

  65. A. Ellis, A. Shahrokni, J.M. Ferryman, PETS 2009 Benchmark Data. Online: http://pets.rdg.ac.uk/pub/PETS2009/Crowd_PETS09_dataset/a_data/Crowd_PETS09/ (2009). Accessed 01 May 2019

  66. Robert Fisher, CAVIAR Test Case Scenarios. Online: http://homepages.inf.ed.ac.uk/rbf/CAVIARDATA1/ (2003). Accessed 01 May 2019

  67. Ryoo, M. S. and Aggarwal, J. K., UT-Interaction Dataset, ICPR contest on Semantic Description of Human Activities (SDHA). Online: http://cvrc.ece.utexas.edu/SDHA2010/Human_Interaction.html (2010). Accessed 01 May 2019

  68. Baqué, P., Fua, P.: EPFL data set: Multi-camera Pedestrian Videos. Online: http://cvlab.epfl.ch/data/pom (2008). Accessed 01 May 2019

  69. Kitware, VIRAT Video Dataset Release 2.0. Online: https://data.kitware.com/#collection/56f56db28d777f753209ba9f (2016). Accessed 01 May 2019

  70. Kumar, P., Singhal, A., Mehta, S., Mittal, A.: Real-time moving object detection algorithm on high-resolution videos using GPUs. J. Real Time Image Process. 11(1), 93–109 (2016)

    Article  Google Scholar 

  71. Benenson, R., Mathias, M., Timofte, R., Van Gool, L.: Pedestrian detection at 100 frames per second. IEEE Conf. Comp. Vis. Pattern Recognit. CVPR 2012, 2903–2910 (2012)

    Google Scholar 

  72. Miyamoto, R., Sugano, H.: Parallel implementation strategy for CoHOG-based pedestrian detection using a multi-core processor. IEICE Trans. Fundam. Electron. Commun. Comp. Sci. 94(11), 2315–2322 (2011)

    Article  Google Scholar 

  73. Bauer, S., Köhler, S., Doll, K., Brunsmann, U.: FPGA-GPU architecture for kernel SVM pedestrian detection. IEEE Comp. Soc. Conf. Comp. Vis. Pattern Recognit. Workshops CVPRW 2010, 61–68 (2010)

    Google Scholar 

  74. Weimer, D., Köhler, S., Hellert, C., Doll, K., Brunsmann, U., Krzikalla, R.: Gpu architecture for stationary multisensor pedestrian detection at smart intersections. IEEE Intell. Veh. Symp. IV 2011, 89–94 (2011)

    Google Scholar 

  75. Lillywhite, K., Lee, D.J., Zhang, D.: Real-time human detection using histograms of oriented gradients on a GPU. IEEE Workshop Appl. Comp. Vis. WACV 2009, 1–6 (2009)

    Google Scholar 

  76. Wojek, C., Dorkó, G., Schulz, A., Schiele, B.: Sliding-windows for rapid object class localization: a parallel technique. In: Rigoll, G. (ed.) DAGM 2008, LNCS 5096, pp. 71–81. Springer, Berlin (2008)

    Google Scholar 

  77. Otsuka, T., Aoki, T., Hosoya, E., Onozawa, A.: An image recognition system for multiple video inputs over a multi-FPGA system. In: IEEE 6th International Symposium on Embedded Multicore Socs, MCSoC 2012, pp. 1–7, September (2012)

  78. Negi, K., Dohi, K., Shibata, Y., Oguri, K.: Deep pipelined one-chip FPGA implementation of a real-time image-based human detection algorithm. Int. Conf. Field Program. Technol. FPT 2011, 1–8 (2011)

    Google Scholar 

  79. Hahnle, M., Saxen, F., Hisung, M., Brunsmann, U, Doll, K.: FPGA-based real-time pedestrian detection on high-resolution images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 629–635 (2013)

Download references

Author information

Authors and Affiliations

  1. Laboratory of Electronics and Microelectronics, Faculty of Sciences of Monastir, University of Monastir, 5000, Monastir, Tunisia

    Haythem Bahri, Marwa Chouchene, Fatma Ezahra Sayadi & Mohamed Atri

  2. College of Computer Science, King Khalid University, Abha, Saudi Arabia

    Mohamed Atri

Authors
  1. Haythem Bahri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marwa Chouchene
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fatma Ezahra Sayadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohamed Atri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haythem Bahri.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bahri, H., Chouchene, M., Sayadi, F.E. et al. Real-time moving human detection using HOG and Fourier descriptor based on CUDA implementation. J Real-Time Image Proc 17, 1841–1856 (2020). https://doi.org/10.1007/s11554-019-00935-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-019-00935-1

Keywords

Search

Navigation