Skip to main content

A fast hybrid computer vision technique for real-time embedded bus passenger flow calculation through camera

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Bus passenger flow calculation system is a critical part of the smart public transportation framework. Bus passenger flow information can help to make data statistics report of the passenger at a bus station which can be used by public transport operator to evaluate the quality of the transportation. Statistics report of crowded passengers in the bus station help managers to understand the bus transit operations, can provide the database for the intelligent transportation scheduling, help to provide more and better services for passengers, overall data statistics of passengers has important practical significance to improve public transport environment. This paper presents a passenger counting algorithm based on hybrid machine learning approach. In the first step, an advanced method is used to extract the Histogram of oriented gradients (HOG) feature of passenger’s heads. Classification of head features is done by using support vector machine (SVM) as a classifier for the liner model. Heads are detected successfully after performing all steps. In next step Kanade-Lucas-Tomasi (KLT) is used to reality head tracking, the multiple target tracking is achieved and the head motion trajectory of passenger target is captured stably. At last, the trajectory is analyzed and the automatic counting of bus passenger flow is realized. In the last step, the proposed algorithm is move to embedded system for practical implementation. In this paper, the algorithm intends to use ADSP-BF609 embedded platform for transplantation. The experimental results demonstrate that the statistical accuracy of the proposed algorithm is enhanced successfully; especially during the daytime with the good illustration, the effective counting of the passenger flow is achieved and the inward and outward passenger counting can be realized. In this paper three feature extraction models are used namely local binary patterns, histograms of oriented gradients and binarized statistical image in order to get accurate features. Furthermore, three common classification techniques including naïve bayes classifier, boosted tress and support vector machines are used for fine classification of extracted vectors obtained from different features extractors model. 94.50% accuracy is achieved when support vector machine (SVM) classifies the features extracted using Histogram of oriented gradients (HOG). SVM surpasses the accuracy obtained by Boosted tree namely 81.30% using Histogram of oriented gradients (HOG) features.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Aizerman MA, Braverman EM, Rozonόer LI (1964) Theoretical foundations of the potential function method in pattern recognition learning. Autom Remote Control 25:821–837

    Google Scholar 

  2. Ang Y-b, Gao H, Zhang M-j (2011) Cascade features based method for pedestrian detection in street scene. Joumal of Computer Applications 3l:129–132

    Google Scholar 

  3. ANTONINI G, MARTINEZ SV, BIERLAIRE M et al (2006) Behavioral priors for detection and tracking of pedestrians in video sequences. Int J Comput Vis 69(2):159–180

    Article  Google Scholar 

  4. Aronszajn N (1950) Theory of reproducing kernels. Trans Am Math Soc 686:337–404

    Article  MathSciNet  Google Scholar 

  5. Barandiaran J, Murguia B, Boto F (2008) Real-time people countin using multiple lines. In: 2008 Ninth International Workshop on Imag Analysis for Multimedia Interactive Services, p 159–162. IEEE

  6. Beltran A, Erickson VL, Cerpa AE (2013) Thermosense: occupanc thermal based sensing for hvac control. In: Proceedings of the 5th ACM Workshop on Embedded Systems For Energy-Efficient Buildings, p 1–8. ACM

  7. Beymer D, Konolige K (1999) Real-time tracking of multiple people using stereo

  8. Brostow GJ, Cipolla R (2006) Unsupervised Bayesian detection of independent motion in crowds. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., vol. 1, p 594–601

  9. Burges C (1998) A tutorial on support vector machines for pattern recognition. In: Data mining and knowledge discovery, vol 2. Kluwer Academic Publishers, Boston

    Google Scholar 

  10. Byrne JA, Gerdes L (2005) The man who invented management. BusinessWeek. Retrieved November 2, 2009

  11. Chen T-Y, Chen Z-X (2006) An intelligent people-flow counting method for passing through a gate. In: 2006 IEEE Conference on Robotics, Automation and Mechatronics, p 1969–2011

  12. Chen T, Chen C, Wang D, Kuo Y (2010) A people counting system based on face-detection. International Conference on Genetic and Evolutionary Computing, p 699–702

  13. Cheng G-t, Chen X, Guo Z-z (2011) Pedestrian detection method of vision based on HOG features. Transducer and Microsystem Technologies 30:68–74

    Google Scholar 

  14. Comaniciu D, Ramesh V (2011). Robust detection and tracking of human faces with an active camera. IEEE Visual Surveillance

  15. Conrad G, Johnsonbaugh R (1994) A real-time people counter. Proceedings of the 1994 ACM symposium on Applied computing, 0–89791–647-6

  16. Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge

    Book  Google Scholar 

  17. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., vol. 2, p 886–893

  18. Dollár P, Babenko B, Belongie S, Perona P, Tu Z (2008) Multiple component learning for object detection. In: Proc. ECCV, p 211–224

  19. Dosovitskiy A, Fischer P, Ilg E, Hausser P, Hazırbas C, Golkov V FlowNet: learning optical flow with convolutional networks. ICCV

  20. Fang WN (2009) Recognizing the passenger number in (tem-ples scenes by RBF neural network). Journal of Beijing Jiro tong University 33(4):29–33

    Google Scholar 

  21. Felzenszwalb P, McAllester D, Ramanan D (2008) A discriminatively trained, multiscale, deformable part model. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., p 1–8

  22. Gao CHQ, Yu DH, Li Q et al (2011) Statistics of pedestrian flows bused on feature matching in video sequenocp. Video Engineering 35(15):20–23

    Google Scholar 

  23. Ge W, Collins RT (2009) Marked point processes for crowd counting. In: Proc. CVPR, p 2913–2920

  24. Haritaoglu I, Flickner M (2001) Detection and tracking of shopping groups in stores. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 0–7695–1272-0

  25. Hashimoto K, Morinaka K, Yoshiike N, Kawaguchi C, Matsueda S (1997) People count system using multi-sensing application, 1997 International conference on solid state sensors and actuators, 0–7803–3829-4

  26. Hashimoto K, Morinaka K, Yoshiike N, Kawaguchi C, Matsueda S (1997) People count system using multi-sensing application, 1997 International conference on solid state sensors and actuators, 0–7803–3829-4

  27. Heckman N (1997) The theory and application of penalized least squares methods or reproducing kernel hilbert spaces made easy

  28. Huanc CHL (2011) The study on counting passengers based on machine vision (1). Huazhong University of Science and Technology, Wuhan, pp 45–66

    Google Scholar 

  29. Huang PC, Lee SS, Kuo YH, Lee KR (2010) A flexible sequence alignment approach on pattern mining and matching for human activity recognition. Expert Syst Appl 37(1):298–306

    Article  Google Scholar 

  30. Hui T-W, Tang X, Loy CC. LiteFlowNet: A Lightweight Convolutional Neural Networkfor Optical Flow Estimation, arXiv:1805.07036v1 [cs.CV]

  31. Ilg E, Mayer N, Saikia T, Keuper M, Dosovitskiy A, Brox T (2019). FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks

  32. Kalas MS (2014) Real Time Face Detection and Tracking Using OPENCV. International Journal of Soft Computing and Artificial Intelligence

  33. Kolekar MH, Bharti N, Patil PN (2016) Detection of fence climbing using activity recognition by support vector machine classifier. In 2016 IEEE Region 10 Conference (TENCON) (p 398–402). Singapore

  34. Küblbeck C, Ernst A (2009) Face detection and tracking in video sequences using the modified census transformation. Electronic Imaging Department, Fraunhofer Institute for Integrated Circuits, Hsin -Chu

    Google Scholar 

  35. Leibe B, Seemann E, Schiele B (2005) Pedestrian detection in crowded scenes. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., vol. 1, p 875–885

  36. Lewis JP. Tutorial on SVM, CGIT Lab, USC

  37. Li XY (2013) On the key research of the statistics of passengers on bus using multi-camera (I). Jilin University, Changchun, pp 26–53

    Google Scholar 

  38. Li J, Huang L, Liu C (2012) People count across multiple cameras for intelligent video surveillance. In: 9th IEEE International Conference on Advanced Video and Signal-Based Surveillance, Beijing, China, p 178–183

  39. Li Y, Huang C, Nevatia R (2009) Learning to associate: Hybrid-Boosted multi-target tracker for crowded scene. In: Proc. IEEE Conf.Comput. Vis. Pattern Recognit., p 2953–2960

  40. Lin S-F, Chen J-Y, Chao H-X (2001) Estimation of number of people in crowded scenes using perspective transformation. IEEE Trans Syst, Man, Cybern 31(6):645–654

    Article  Google Scholar 

  41. Liu H, Qian Y, Lin S (2010) Detecting persons using Hough circle transform in surveillance video. In: 2010 International Conference on Computer Vision Theory and Applications, p 267–270

  42. Marcenaro CRL, Morerio P (2012) Performance evaluation of multicamera visual tracking. In: 9th IEEE International Conference on Advanced Video and Signal-Based Surveillance, Beijing, China, p 464–469

  43. Masoud O, Papanikolopoulos NT (2001) A novel method for tracking and counting pedestrians in real-time using a single camera. IEEE Trans on Vehicular Tech 50:1267–1278

    Article  Google Scholar 

  44. Matsuyama T, Wada T, Habe H, Tanahashi K (2001) Background subtraction under varying illumination. Trans, IEICE J84-D-II:2201–2211

    Google Scholar 

  45. Ming Y, Wei L (2009) A new method for passenger flow counting system based on surveillance video. In: Proceedings of the 2nd International Conference on Intelligent Networks and Intelligent System, p 453–456

  46. Mitchell T (1997) Machine Learning. McGraw-Hill Computer science series

  47. Mitchell T (1997) Machine Learning. McGraw-Hill Computer science series

  48. Morerio CRP, Marcenaro L (2012) People count estimation on small crowds. In: 9th IEEE International Conference on Advanced Video and Signal-Based Surveillance, Beijing, China, p 476–480

  49. Morris BT, Trivedi MM (2008) A survey of vision-based trajectory learning and analysis for surveillance. IEEE Trans Circuits Syst Video Technol 18(8):1114–1127 Special Issue Video Surveillance

    Article  Google Scholar 

  50. Munir S, Arora R, Hesling C, Li J, Francis J, Shelton C, Martin C, Rowe A, Berges M (2017) Real-time fine grained occupancy estimation using depth sensors on ARM embedded platforms. In: Proceedings of the 23rd IEEE Real Time and Embedded Technology and Application Symposium

  51. Oliveira LES, Mansano M, Koerich AL, Britto AS Jr (2011) 2d principal component analysis for face and facial-expression recognition. Computing in Science & Engineering 13(3):9–13

    Article  Google Scholar 

  52. Pane C, Gasparini M, Prati A, Gualdi G, Cucchiara R (2013) A people counting system for business analytics. In: 10th IEEE International Conference on Advanced Video and Signal-Based Surveillance, Krakow, Poland, p 135–140

  53. HH Park, HG Lee, S-I Noh, J Kim (2006) An area-based decision rule for people-counting systems. In: Lecture Notes in Computer Science, p 450–457

    Chapter  Google Scholar 

  54. Rabaud V, Belongie SJ (2006) Counting crowded moving objects. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit, p 705–711

  55. Rossi M, Bozzoli A (1994) Tracking and counting moving people. IEEE Proc of Int Conf Image Processing 3:212–216

    Article  Google Scholar 

  56. Schofield AJ, Stonham TJ, Metha PA (1995) A RAM based neural network approach to people counting, Fifth International Conference on Image Processing and its Applications, 0–85296–642-3

  57. Segen J, Pingali S (1995) A camera based system for tracking people in real time. IEEE Proc of Int Conf Pattern Recognition 3:63–67

    Google Scholar 

  58. Segen J, Pingali SG (1996) A camera-based system for tracking people in real time. Proceedings of the 13th International Conference on Pattern Recognition, 0–8186-7282-X

  59. Sexton G, Zhang X, Redpath D, Greaves D (1995) Advances in automated pedestrian counting, European Convention on Security and Detection, 0–85296–640-7

  60. Shio A, Sklansky J (1991) Segmentation of people in motion, 1991 Proceedings of the IEEE workshop on Visual Motion, 0–8186–2153-2

  61. Skapura DM (1996) Building neural networks. ACM press, New York

    Google Scholar 

  62. Talu MF, Turkoglu I, Cebeci M (2012) A hybrid tracking method for scaled and oriented objects in crowded scenes, Expert systems with applications. 38(11):13 682–13 687

  63. Terada K, Matsubara K (2009) A method for counting multidirection passerby by using circular space-time image. IEEJ Trans EIS 129(6)

    Article  Google Scholar 

  64. Terada K, Yoshida D, Oe S, Yamaguchi J (1999) A method of counting the passing people by using the stereo images. International conference on image processing, 0–7803–5467-2

  65. Terada K, Yoshida D, Oe S, Yamaguchi J (1999) A method of counting the passing people by using the stereo images, International conference on image processing, 0–7803–5467-2

  66. Tesei A, Teschioni A, Regazzoni CS, Vernazza G (1996) Long-Memory" matching of interacting complex objects from real image sequences, 1996 Conference on Time Varying Image Processing and Moving Objects Recognition

  67. Tu PH, Rittscher J, Perera A, Krahnstoever N (2005) Detecting and countingpeople in surveillance applications. In: IEEE International Conference on Advanced Video and Signal Based Surveillance Proceedings of AVSS 2005, p 306–311

  68. Tutorial slides by Andrew Moore (2019). Http://www.cs.cmu.edu/~awm

  69. Vapnik V (1995) The nature of statistical learning theory. Springer, New York ISBN 0–387–94559-8

    Book  Google Scholar 

  70. Vapnik V, Golowich S, Smola A (1997) Support vector method for function approximation, regression estimation, and signal processing. In: Mozer M, Jordan M, Petsche T (eds) Advances in neural information processing systems, vol 9. MIT Press, Cambridge, pp 281–287

    Google Scholar 

  71. Viola P, Jones M, Snow D (2005) Detecting pedestrians using patterns of motion and appearance. Int J Comput Vis 63(2):153–161

    Article  Google Scholar 

  72. Wikipedia Online (2019). Http://en.wikipedia.org/wiki

  73. Wu Y (2008) A new NN-SVM algorithm based on girds [13]. Journal of Chongqing University of Posts and Telecommunications: Natural Science Edition 20(6):706–709

    Google Scholar 

  74. Wu B, Nevatia R (2005) Detection of multiple, partially occluded humans in a single image by Bayesian combination of edgelet part detectors. In: Proc. IEEE Int. Conf. Comput. Vis., vol. 1, p 90–97

  75. Xiao J, Hays J, Ehinger K (2010) Sun database: Large-scale scene recognition from abbey to zoo. Computer Vision and Pattern Recognition (CVPR)

  76. Xu H, Lv P, Meng L (2010) A people counting system based on head-shoulder detection and tracking insurveillance video. In: 2010 International Conference on Computer Design and Applications, p V1394–V1398

  77. Yang Y, Ramanan D (2011) Articulated pose estimation with exible mixtures-of-parts. Computer Vision and Pattern Recognition (CVPR)

  78. Ye W, Xu Y, Zhong Z (2007) Robust people counting in crowded environment. In: 2007 IEEE International Conference on Robotics and Biomimetics, p 1133–1137

  79. Yu HB, Liu L (2008) A vision-based method to estimate passenger flow in bus. Journal of Image and Graphics (15):716–722

  80. Zavaschi THH, Britto AS Jr, Oliveira LES, Koerich AL (2013) Fusion of feature sets and classifiers for facial expression recognition. Expert Syst Appl 40(2):646–655

    Article  Google Scholar 

  81. Zhang E, Chen F (2007) A fast and robust people counting method in video surveillance. In: Proceedings of 2007 International Conference on Computational Intelligence and Security, p 339–343

  82. Zhang X, Gao Y (2009) Face recognition across pose: A review. Pattern Recogn

  83. Zhang D, Guo G, Huang D, Han J PoseFlow: a deep motion representation for understanding human behaviors in videos. CVPR

  84. Zhang D, Han J SPFTN: A Joint Learning Framework for Localizing and Segmenting Objects in Weakly Labeled Videos. IEEE Trans Pattern Anal Mach Intell

  85. Zhao T, Nevatia R (2003) Bayesian human segmentation in crowded situations. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., vol. 2, p 459–466

  86. Zhao T, Nevatia R (2004) Tracking multiple humans in crowded environment. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., p II-406–II-413

  87. Zhao T, Nevatia R, Wu B (2008) Segmentation and tracking of multiple humans in crowded environments. IEEE Trans Pattern AnalMach Intell 30(7):1198–1211

    Article  Google Scholar 

  88. Zheng Q, Yang M (2017) A video stabilization method based on inter-frame image matching score. Glob J Comput Sci Technol 17:41–46

    Google Scholar 

  89. Zhu AZ, Yuan L, Chaney K, Daniilidis K. EV-FlowNet: self-supervised optical flow estimation for event-based cameras, https://arxiv.org/abs/1802.06898

  90. Zulkifley MA, Moran B (2012) Robust hierarchical multiple hypothesis tracker for multiple-object tracking. Expert Syst Appl 39(16):1231912331

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xu Huarong.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Haq, E.U., Huarong, X., Xuhui, C. et al. A fast hybrid computer vision technique for real-time embedded bus passenger flow calculation through camera. Multimed Tools Appl 79, 1007–1036 (2020). https://doi.org/10.1007/s11042-019-08167-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08167-y

Keywords