Abstract
Real-time applications of image and video processing algorithms have seen explosive growth in number and complexity over the past decade driven by consumer, scientific and defense applications exploiting inexpensive digital video cameras and networked computing device. This growth has opened up different alternatives to greatly enhance the surveillance capabilities using new architectures and parallelization strategies developed due to the increased accessibility of multicore, multi-threaded processors along with general purpose graphics processing units (GPUs). In this paper, we present a new implementation of a moving human detection algorithm on GPU based on the programming language CUDA. In our approach, the moving object is extracted by background subtraction based on the GMM (Gaussian Mixture Model) on GPU. Then, two complementary features are extracted for moving object classification. They are contour-based description: FD or Fourier Descriptor and region-based description: HOG or Histogram of Oriented Gradient. Both descriptors will then be effectively integrated to SVM (Support Vector Machine), which is able to provide the posterior probability, to achieve better performance. The implementation of such algorithm on a GPU allows a great performance in terms of execution time since it is 19 times faster than that on a CPU. Experimental results show also that the proposed approach outperforms some existing techniques and can detect pedestrians in real-time effectively.












Similar content being viewed by others
References
Shashua, A., Gdalyahu, Y., Hayun, G.: Pedestrian detection for driving assistance systems: single-frame classification and system level performance. IEEE Intell. Veh. Symp. IV 2004, 1–6 (2004)
Zhao, L., Thorpe, C.E.: Stereo-and neural network-based pedestrian detection. IEEE Trans. Intell. Transp. Syst. 1(3), 148–154 (2000)
Viola, P., Jones, M.J., Snow, D.: Detecting pedestrians using patterns of motion and appearance. Int. J. Comp. Vis. 63(2), 153–161 (2005)
Papageorgiou, C., Poggio, T.: Trainable pedestrian detection. In: Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348), Kobe, vol. 4, pp. 35–39 (1999). https://doi.org/10.1109/ICIP.1999.819462
Hogg, D.: Model-based vision: a program to see a walking person. Image Vis. Comput. 1(1), 5–20 (1983)
Guo, Y., Xu, G., Tsuji, S.: Understanding human motion patterns. In: Proceedings of the 12th IAPR International Conference on Pattern Recognition, vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5), vol. 2, pp. 325–329 (1994). https://doi.org/10.1109/ICPR.1994.576929
Rohr, K.: Towards model-based recognition of human movements in image sequences. CVGIP Image Underst. 59(1), 94–115 (1994)
Szarvas, M., Yoshizawa, A., Yamamoto, M., Ogata, J.: Pedestrian detection with convolutional neural networks. In: IEEE Proceedings. Intelligent Vehicles Symposium, Las Vegas, NV, USA, pp. 224–229 (2005). https://doi.org/10.1109/IVS.2005.1505106
Zhu, Q., Yeh, M.C., Cheng, K.T., Avidan, S.: Fast human detection using a cascade of histograms of oriented gradients. IEEE Comp. Soc. Conf. Comp. Vis. Pattern Recognit. 2, 1491–1498 (2006)
Banerjee, P., Sengupta, S.: Human motion detection and tracking for video surveillance. In: Proceedings of the National Conference of Communications, IIT Bombay, Mumbai, pp. 88–92 (2008)
Wang, X., Han, T.X., Yan, S.: An HOG-LBP human detector with partial occlusion handling. In: 2009 IEEE 12th International Conference on Computer Vision, Kyoto, pp. 32–39 (2009). https://doi.org/10.1109/ICCV.2009.5459207
Bolme, D.S., Lui, Y.M., Draper, B.A., Beveridge, J.R.: Simple real-time human detection using a single correlation filter. In: 2009 12th IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, Snowbird, UT, pp. 1–8 (2009). https://doi.org/10.1109/PETS-WINTER.2009.5399555
DeCann, B., Ross, A.: Gait curves for human recognition, backpack detection, and silhouette correction in a nighttime environment. In: Proc. SPIE 7667, Biometric Technology for Human Identification VII, 76670Q (2010). https://doi.org/10.1117/12.851296
Barnich, O.: Motion detection and human recognition in video sequences, Thesis report, Faculty of Engineering and Computer Science, University of Liège (2010)
Nguyen, D.T., Li, W., Ogunbona, P.O.: Human detection from images and videos: a survey. Pattern Recognit. 51, 148–175 (2016)
Thanh N.D.: Human detection from images and video. Thesis report, College of Engineering and Computer Science, University of Central Florida—Orlando, FL (2012)
Paul, M., Haque, S.M., Chakraborty, S.: Human detection in surveillance videos and its applications—a review. EURASIP J. Adv. Sig. Process. 2013(1), 176 (2013)
Chesnais T.: Contextualization of a pedestrian detector: Application to the surveillance of public spaces. Thesis report, Blaise Pascal University, Clermont-Ferrand II (2013)
Ouyang, W., Wang, X.: Joint deep learning for pedestrian detection. In: 2013 IEEE International Conference on Computer Vision, Sydney, NSW, pp. 2056–2063 (2013). https://doi.org/10.1109/ICCV.2013.257
Dehghan, A., Idrees, H., Zamir, A.R., Shah, M.: Automatic detection and tracking of pedestrians in videos with various crowd densities. In: Weidmann, U., Kirsch, U., Schreckenberg, M. (eds.) Pedestrian and Evacuation Dynamics 2012, pp. 3–19. Springer, Cham (2014)
Bourdev, L.D., Yang, F., Fergus, R.: Deep poselets for human detection. CoRR. arXiv preprint arXiv:1407.0717 (2014)
Mahapatra, A., Mishra, T.K., Sa, P.K., Majhi, B.: Human recognition system for outdoor videos using Hidden Markov model. AEU Int. J. Electron. Commun. 68(3), 227–236 (2014)
Luo, P., Tian, Y., Wang, X., Tang, X.: Switchable deep network for pedestrian detection. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, pp. 899–906 (2014). https://doi.org/10.1109/CVPR.2014.120
Lu, Y., Boukharouba, K., Boonært, J., Fleury, A., Lecoeuche, S.: Application of an incremental SVM algorithm for on-line human recognition from video surveillance using texture and color features. Neurocomput. J. 126, 132–140 (2014)
Wicaksono, I.B., An, F., Mattausch, H.J.: Memory-based hardware-accelerated system for high-speed human detection. Adv. Robot. 28(5), 317–327 (2014)
Emami, A.: Occlusion Handling in Video Surveillance Systems. Thesis report, Faculty of Engineering, Architecture and Information Technology, University of Queensland, (2015)
Hatto, M.: Acceleration of pedestrian detection system using hardware-software co-design. MSc Thesis, Lund University (2015)
Jiang, Y., Ma, J.: Combination features and models for human detection. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, pp. 240–248 (2015). https://doi.org/10.1109/CVPR.2015.7298620
Angelova, A., Krizhevsky, A., Vanhoucke, V., Ogale, A.S., Ferguson, D.: Real-time pedestrian detection with deep network cascades. In: The British Machine Vision Conference, BMVC 2015, vol. 2, pp. 4–16, September (2015)
Ramin, M.: Improvements to tracking pedestrians in video streams using a pre-trained convolutional neural network. Electronic Thesis and Dissertation Repository. 3886 (2016). https://ir.lib.uwo.ca/etd/3886
Ribeiro, D., Mateus, A., Miraldo, P., Nascimento, J.C.: A real-time pedestrian detector using deep learning for human-aware navigation. In: 2017 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), Coimbra, pp. 165–171 (2017). https://doi.org/10.1109/ICARSC.2017.7964070
Suleiman, A., Sze, V.: An energy-efficient hardware implementation of HOG-based object detection at 1080HD 60 fps with multi-scale support. J. Sig. Process. Syst. 84(3), 325–337 (2016)
Campmany, V., Silva, S., Espinosa, A., Moure, J.C., Vázquez, D., López, A.M.: GPU-based pedestrian detection for autonomous driving. Procedia. Comput. Sci. 80, 2377–2381 (2016)
Zhang, M., Xin, M.: Human detection using random color similarity feature and random ferns classifier. PLoS One J. 11(9), e0162830 (2016)
Lee, N., Weng, X., Boddeti, V.N., Zhang, Y., Beainy, F., Kitani, K., Kanade, T.: Visual compiler: synthesizing a scene-specific pedestrian detector and pose estimator. arXiv preprint arXiv:1612.05234 (2016)
Kim, J.H., Hong, H.G., Park, K.R.: Convolutional neural network-based human detection in nighttime images using visible light camera sensors. Sensors 17(5), 1065 (2017)
AlDahoul, N., Sabri, M., Qalid, A., Mansoor, A.M.: Real-time human detection for aerial captured video sequences via deep models. Comput. Intell. Neurosci. 2018, 14 (2018). https://doi.org/10.1155/2018/1639561
Almonfrey, D., do Carmo, A.P., de Queiroz, F.M., Picoreti, R., Vassallo, R.F., Salles, E.O.T.: A flexible human detection service suitable for Intelligent Spaces based on a multi-camera network. Int. J. Distrib. Sensor Netw. 14(3), 1550147718763550 (2018)
Afifi, M., Ali, Y., Amer, K., Shaker, M., ElHelw, M.: Robust real-time pedestrian detection in aerial imagery on Jetson TX2. arXiv preprint arXiv:1905.06653 (2019)
Vandersteegen, M., Van Beeck, K., Goedemé, T.: Real-time multispectral pedestrian detection with a single-pass deep neural network. International Conference Image Analysis and Recognition, pp. 419–426. Springer, Cham (2018)
Permuter, H., Francos, J., Jermyn, I.H.: Gaussian mixture models of texture and colour for image database retrieval. In: 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings (ICASSP ’03), Hong Kong, pp. III-569 (2003). https://doi.org/10.1109/ICASSP.2003.1199538
Liu, T., Stathaki, T.: Faster R-CNN for robust pedestrian detection using semantic segmentation network. Front Neurorobot 12, 64 (2018)
Mao, J., Xiao, T., Jiang, Y. Cao, Z.: What can help pedestrian detection?. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3127–3136 (2017)
Bahri, H., Chouchene, M., Khemiri, R., Sayadi, F., Atri, M.: March. fast moving human detection using fourier and HOG descriptors based CUDA. In: IEEE 15th International Multi-Conference on Systems, Signals & Devices (SSD), pp. 202–207 (2018)
Flohr, F., Gavrila, D.M.: PedCut: an iterative framework for pedestrian segmentation combining shape models and multiple data cues. In: BMVC (2013)
Jeannin, S., Bober, M.: Description of core experiments for mpeg-7 motion/shape. MPEG-7, ISO/IEC/JTC1/SC29/WG11/MPEG99/N2690, Seoul (1999)
Zivkovic, Z.: Improved adaptive Gaussian mixture model for background subtraction. In: Proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004, vol. 2, pp. 28–31, August (2004)
Zivkovic, Z., Van Der Heijden, F.: Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recognit. Lett. 27(7), 773–780 (2006)
Pham, V., Vo, P., Hung, V.T.: GPU implementation of extended gaussian mixture model for background subtraction. IEEE Int. Conf. Comput. Commun. Technol. Res. Innov. Vis. Future 2010, 1–4 (2010)
Dariu M. Gavrila, PedCut 2013 Segmentation Dataset. Online: http://www.lookingatpeople.com/download-daimler-ped-segm-benchmark/index.html (2013). Accessed 01 May 2019
Richard Ralph, MPEG-7 Core Experiment CE-Shape-1 Test Set. Online: http://www.dabi.temple.edu/~shape/MPEG7/dataset.html (1999). Accessed 01 May 2019
Rui Zhao, CUHK01 Dataset. Online: http://www.ee.cuhk.edu.hk/~rzhao/ (2017). Accessed 01 May 2019
Mary Pat Fitzgerald, Pedestrian Data. Online: http://cbcl.mit.edu/software-datasets/PedestrianData.html (2000). Accessed 01 May 2019
Navneet Dalal, INRIA Person Dataset. Online: http://pascal.inrialpes.fr/data/human/ (2016). Accessed 01 May 2019
Bahri, H., Sayadi, F., Khemiri, R., Chouchene, M., Atri, M.: Image feature extraction algorithm based on CUDA architecture: case study GFD and GCFD. IET Comput. Dig. Techniq. 11(4), 125–132 (2017)
Pedersoli, M.; Gonzàlez i Sabaté, J., dir.; Roca, X.: Hierarchical multiresolution models for fast object detection. [Barcelona]: Universitat Autònoma de Barcelona, 2015. 1 recurs electrònic (139 p.). ISBN 9788449032066. Tesi doctoral, Departament de Ciències de la Computació, Universitat Autònoma de Barcelona (2012) [Checked: 5 decsember 2019]
Smach, F., Miteran, J., Atri, M., et al.: An FPGA-based accelerator for Fourier descriptors computing for color object recognition using SVM. J. Real Time Image Process 2(4), 249–258 (2007)
NVIDIA, C., CUDA Occupancy Calculator. CUDA SDK. Online: https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwjOhIu8L7XAhXQYlAKHWJuA5QQFggmMAA&url=https%3A%2F%2Fdeveloper.download.nvidia.com%2Fcompute%2Fcuda%2FCUDA_Occupancy_calculator.xls&usg=AOvVaw3C1_WHEkOfxeH1sjzxGYB5 (2010). Accessed 01 May 2019
Davis, J.W., Sharma, V.: Background-subtraction using contour-based fusion of thermal and visible imagery. Comput. Vis. Image Underst. 106, 162–182 (2007)
Sudowe, P., Leibe, B.: Efficient Use of Geometric Constraints for Sliding-Window Object Detection in Video. In IEEE 8th International Conference on Computer Vision Systems, ICVS 2011, Sophia, pp. 11–20, September (2011)
Pedersoli, M., Gonzàlez i Sabaté, J., Roca, X.: Hierarchical multiresolution models for fast object detection, University Autònoma of Barcelona. 1 recurs electrònic (139 p.). ISBN 9788449032066. Thesis doctoral report—Universitat Autònoma de Barcelona. Departament de Ciències de la Computació. Online: https://ddd.uab.cat/record/130257(2012). Accessed 01 May 2019
Ahmed Magdi Osman, GPU-HOG. Online: https://github.com/ahmedmagdiosman/GPU-HOG (2015). Accessed 01 May 2019
Fleuret, F., Berclaz, J., Lengagne, R., Fua, P.: Multicamera people tracking with a probabilistic occupancy map. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 267–282 (2008). https://doi.org/10.1109/TPAMI.2007.1174
Alex Leykin, Dataset 03: OSU Color and Thermal Database. Online: http://vcipl-okstate.org/pbvs/bench/Data/03/download.html (2007). Accessed 01 May 2019
A. Ellis, A. Shahrokni, J.M. Ferryman, PETS 2009 Benchmark Data. Online: http://pets.rdg.ac.uk/pub/PETS2009/Crowd_PETS09_dataset/a_data/Crowd_PETS09/ (2009). Accessed 01 May 2019
Robert Fisher, CAVIAR Test Case Scenarios. Online: http://homepages.inf.ed.ac.uk/rbf/CAVIARDATA1/ (2003). Accessed 01 May 2019
Ryoo, M. S. and Aggarwal, J. K., UT-Interaction Dataset, ICPR contest on Semantic Description of Human Activities (SDHA). Online: http://cvrc.ece.utexas.edu/SDHA2010/Human_Interaction.html (2010). Accessed 01 May 2019
Baqué, P., Fua, P.: EPFL data set: Multi-camera Pedestrian Videos. Online: http://cvlab.epfl.ch/data/pom (2008). Accessed 01 May 2019
Kitware, VIRAT Video Dataset Release 2.0. Online: https://data.kitware.com/#collection/56f56db28d777f753209ba9f (2016). Accessed 01 May 2019
Kumar, P., Singhal, A., Mehta, S., Mittal, A.: Real-time moving object detection algorithm on high-resolution videos using GPUs. J. Real Time Image Process. 11(1), 93–109 (2016)
Benenson, R., Mathias, M., Timofte, R., Van Gool, L.: Pedestrian detection at 100 frames per second. IEEE Conf. Comp. Vis. Pattern Recognit. CVPR 2012, 2903–2910 (2012)
Miyamoto, R., Sugano, H.: Parallel implementation strategy for CoHOG-based pedestrian detection using a multi-core processor. IEICE Trans. Fundam. Electron. Commun. Comp. Sci. 94(11), 2315–2322 (2011)
Bauer, S., Köhler, S., Doll, K., Brunsmann, U.: FPGA-GPU architecture for kernel SVM pedestrian detection. IEEE Comp. Soc. Conf. Comp. Vis. Pattern Recognit. Workshops CVPRW 2010, 61–68 (2010)
Weimer, D., Köhler, S., Hellert, C., Doll, K., Brunsmann, U., Krzikalla, R.: Gpu architecture for stationary multisensor pedestrian detection at smart intersections. IEEE Intell. Veh. Symp. IV 2011, 89–94 (2011)
Lillywhite, K., Lee, D.J., Zhang, D.: Real-time human detection using histograms of oriented gradients on a GPU. IEEE Workshop Appl. Comp. Vis. WACV 2009, 1–6 (2009)
Wojek, C., Dorkó, G., Schulz, A., Schiele, B.: Sliding-windows for rapid object class localization: a parallel technique. In: Rigoll, G. (ed.) DAGM 2008, LNCS 5096, pp. 71–81. Springer, Berlin (2008)
Otsuka, T., Aoki, T., Hosoya, E., Onozawa, A.: An image recognition system for multiple video inputs over a multi-FPGA system. In: IEEE 6th International Symposium on Embedded Multicore Socs, MCSoC 2012, pp. 1–7, September (2012)
Negi, K., Dohi, K., Shibata, Y., Oguri, K.: Deep pipelined one-chip FPGA implementation of a real-time image-based human detection algorithm. Int. Conf. Field Program. Technol. FPT 2011, 1–8 (2011)
Hahnle, M., Saxen, F., Hisung, M., Brunsmann, U, Doll, K.: FPGA-based real-time pedestrian detection on high-resolution images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 629–635 (2013)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Bahri, H., Chouchene, M., Sayadi, F.E. et al. Real-time moving human detection using HOG and Fourier descriptor based on CUDA implementation. J Real-Time Image Proc 17, 1841–1856 (2020). https://doi.org/10.1007/s11554-019-00935-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11554-019-00935-1