Abstract
Bus passenger flow calculation system is a critical part of the smart public transportation framework. Bus passenger flow information can help to make data statistics report of the passenger at a bus station which can be used by public transport operator to evaluate the quality of the transportation. Statistics report of crowded passengers in the bus station help managers to understand the bus transit operations, can provide the database for the intelligent transportation scheduling, help to provide more and better services for passengers, overall data statistics of passengers has important practical significance to improve public transport environment. This paper presents a passenger counting algorithm based on hybrid machine learning approach. In the first step, an advanced method is used to extract the Histogram of oriented gradients (HOG) feature of passenger’s heads. Classification of head features is done by using support vector machine (SVM) as a classifier for the liner model. Heads are detected successfully after performing all steps. In next step Kanade-Lucas-Tomasi (KLT) is used to reality head tracking, the multiple target tracking is achieved and the head motion trajectory of passenger target is captured stably. At last, the trajectory is analyzed and the automatic counting of bus passenger flow is realized. In the last step, the proposed algorithm is move to embedded system for practical implementation. In this paper, the algorithm intends to use ADSP-BF609 embedded platform for transplantation. The experimental results demonstrate that the statistical accuracy of the proposed algorithm is enhanced successfully; especially during the daytime with the good illustration, the effective counting of the passenger flow is achieved and the inward and outward passenger counting can be realized. In this paper three feature extraction models are used namely local binary patterns, histograms of oriented gradients and binarized statistical image in order to get accurate features. Furthermore, three common classification techniques including naïve bayes classifier, boosted tress and support vector machines are used for fine classification of extracted vectors obtained from different features extractors model. 94.50% accuracy is achieved when support vector machine (SVM) classifies the features extracted using Histogram of oriented gradients (HOG). SVM surpasses the accuracy obtained by Boosted tree namely 81.30% using Histogram of oriented gradients (HOG) features.
Similar content being viewed by others
References
Aizerman MA, Braverman EM, Rozonόer LI (1964) Theoretical foundations of the potential function method in pattern recognition learning. Autom Remote Control 25:821–837
Ang Y-b, Gao H, Zhang M-j (2011) Cascade features based method for pedestrian detection in street scene. Joumal of Computer Applications 3l:129–132
ANTONINI G, MARTINEZ SV, BIERLAIRE M et al (2006) Behavioral priors for detection and tracking of pedestrians in video sequences. Int J Comput Vis 69(2):159–180
Aronszajn N (1950) Theory of reproducing kernels. Trans Am Math Soc 686:337–404
Barandiaran J, Murguia B, Boto F (2008) Real-time people countin using multiple lines. In: 2008 Ninth International Workshop on Imag Analysis for Multimedia Interactive Services, p 159–162. IEEE
Beltran A, Erickson VL, Cerpa AE (2013) Thermosense: occupanc thermal based sensing for hvac control. In: Proceedings of the 5th ACM Workshop on Embedded Systems For Energy-Efficient Buildings, p 1–8. ACM
Beymer D, Konolige K (1999) Real-time tracking of multiple people using stereo
Brostow GJ, Cipolla R (2006) Unsupervised Bayesian detection of independent motion in crowds. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., vol. 1, p 594–601
Burges C (1998) A tutorial on support vector machines for pattern recognition. In: Data mining and knowledge discovery, vol 2. Kluwer Academic Publishers, Boston
Byrne JA, Gerdes L (2005) The man who invented management. BusinessWeek. Retrieved November 2, 2009
Chen T-Y, Chen Z-X (2006) An intelligent people-flow counting method for passing through a gate. In: 2006 IEEE Conference on Robotics, Automation and Mechatronics, p 1969–2011
Chen T, Chen C, Wang D, Kuo Y (2010) A people counting system based on face-detection. International Conference on Genetic and Evolutionary Computing, p 699–702
Cheng G-t, Chen X, Guo Z-z (2011) Pedestrian detection method of vision based on HOG features. Transducer and Microsystem Technologies 30:68–74
Comaniciu D, Ramesh V (2011). Robust detection and tracking of human faces with an active camera. IEEE Visual Surveillance
Conrad G, Johnsonbaugh R (1994) A real-time people counter. Proceedings of the 1994 ACM symposium on Applied computing, 0–89791–647-6
Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., vol. 2, p 886–893
Dollár P, Babenko B, Belongie S, Perona P, Tu Z (2008) Multiple component learning for object detection. In: Proc. ECCV, p 211–224
Dosovitskiy A, Fischer P, Ilg E, Hausser P, Hazırbas C, Golkov V FlowNet: learning optical flow with convolutional networks. ICCV
Fang WN (2009) Recognizing the passenger number in (tem-ples scenes by RBF neural network). Journal of Beijing Jiro tong University 33(4):29–33
Felzenszwalb P, McAllester D, Ramanan D (2008) A discriminatively trained, multiscale, deformable part model. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., p 1–8
Gao CHQ, Yu DH, Li Q et al (2011) Statistics of pedestrian flows bused on feature matching in video sequenocp. Video Engineering 35(15):20–23
Ge W, Collins RT (2009) Marked point processes for crowd counting. In: Proc. CVPR, p 2913–2920
Haritaoglu I, Flickner M (2001) Detection and tracking of shopping groups in stores. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 0–7695–1272-0
Hashimoto K, Morinaka K, Yoshiike N, Kawaguchi C, Matsueda S (1997) People count system using multi-sensing application, 1997 International conference on solid state sensors and actuators, 0–7803–3829-4
Hashimoto K, Morinaka K, Yoshiike N, Kawaguchi C, Matsueda S (1997) People count system using multi-sensing application, 1997 International conference on solid state sensors and actuators, 0–7803–3829-4
Heckman N (1997) The theory and application of penalized least squares methods or reproducing kernel hilbert spaces made easy
Huanc CHL (2011) The study on counting passengers based on machine vision (1). Huazhong University of Science and Technology, Wuhan, pp 45–66
Huang PC, Lee SS, Kuo YH, Lee KR (2010) A flexible sequence alignment approach on pattern mining and matching for human activity recognition. Expert Syst Appl 37(1):298–306
Hui T-W, Tang X, Loy CC. LiteFlowNet: A Lightweight Convolutional Neural Networkfor Optical Flow Estimation, arXiv:1805.07036v1 [cs.CV]
Ilg E, Mayer N, Saikia T, Keuper M, Dosovitskiy A, Brox T (2019). FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Kalas MS (2014) Real Time Face Detection and Tracking Using OPENCV. International Journal of Soft Computing and Artificial Intelligence
Kolekar MH, Bharti N, Patil PN (2016) Detection of fence climbing using activity recognition by support vector machine classifier. In 2016 IEEE Region 10 Conference (TENCON) (p 398–402). Singapore
Küblbeck C, Ernst A (2009) Face detection and tracking in video sequences using the modified census transformation. Electronic Imaging Department, Fraunhofer Institute for Integrated Circuits, Hsin -Chu
Leibe B, Seemann E, Schiele B (2005) Pedestrian detection in crowded scenes. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., vol. 1, p 875–885
Lewis JP. Tutorial on SVM, CGIT Lab, USC
Li XY (2013) On the key research of the statistics of passengers on bus using multi-camera (I). Jilin University, Changchun, pp 26–53
Li J, Huang L, Liu C (2012) People count across multiple cameras for intelligent video surveillance. In: 9th IEEE International Conference on Advanced Video and Signal-Based Surveillance, Beijing, China, p 178–183
Li Y, Huang C, Nevatia R (2009) Learning to associate: Hybrid-Boosted multi-target tracker for crowded scene. In: Proc. IEEE Conf.Comput. Vis. Pattern Recognit., p 2953–2960
Lin S-F, Chen J-Y, Chao H-X (2001) Estimation of number of people in crowded scenes using perspective transformation. IEEE Trans Syst, Man, Cybern 31(6):645–654
Liu H, Qian Y, Lin S (2010) Detecting persons using Hough circle transform in surveillance video. In: 2010 International Conference on Computer Vision Theory and Applications, p 267–270
Marcenaro CRL, Morerio P (2012) Performance evaluation of multicamera visual tracking. In: 9th IEEE International Conference on Advanced Video and Signal-Based Surveillance, Beijing, China, p 464–469
Masoud O, Papanikolopoulos NT (2001) A novel method for tracking and counting pedestrians in real-time using a single camera. IEEE Trans on Vehicular Tech 50:1267–1278
Matsuyama T, Wada T, Habe H, Tanahashi K (2001) Background subtraction under varying illumination. Trans, IEICE J84-D-II:2201–2211
Ming Y, Wei L (2009) A new method for passenger flow counting system based on surveillance video. In: Proceedings of the 2nd International Conference on Intelligent Networks and Intelligent System, p 453–456
Mitchell T (1997) Machine Learning. McGraw-Hill Computer science series
Mitchell T (1997) Machine Learning. McGraw-Hill Computer science series
Morerio CRP, Marcenaro L (2012) People count estimation on small crowds. In: 9th IEEE International Conference on Advanced Video and Signal-Based Surveillance, Beijing, China, p 476–480
Morris BT, Trivedi MM (2008) A survey of vision-based trajectory learning and analysis for surveillance. IEEE Trans Circuits Syst Video Technol 18(8):1114–1127 Special Issue Video Surveillance
Munir S, Arora R, Hesling C, Li J, Francis J, Shelton C, Martin C, Rowe A, Berges M (2017) Real-time fine grained occupancy estimation using depth sensors on ARM embedded platforms. In: Proceedings of the 23rd IEEE Real Time and Embedded Technology and Application Symposium
Oliveira LES, Mansano M, Koerich AL, Britto AS Jr (2011) 2d principal component analysis for face and facial-expression recognition. Computing in Science & Engineering 13(3):9–13
Pane C, Gasparini M, Prati A, Gualdi G, Cucchiara R (2013) A people counting system for business analytics. In: 10th IEEE International Conference on Advanced Video and Signal-Based Surveillance, Krakow, Poland, p 135–140
HH Park, HG Lee, S-I Noh, J Kim (2006) An area-based decision rule for people-counting systems. In: Lecture Notes in Computer Science, p 450–457
Rabaud V, Belongie SJ (2006) Counting crowded moving objects. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit, p 705–711
Rossi M, Bozzoli A (1994) Tracking and counting moving people. IEEE Proc of Int Conf Image Processing 3:212–216
Schofield AJ, Stonham TJ, Metha PA (1995) A RAM based neural network approach to people counting, Fifth International Conference on Image Processing and its Applications, 0–85296–642-3
Segen J, Pingali S (1995) A camera based system for tracking people in real time. IEEE Proc of Int Conf Pattern Recognition 3:63–67
Segen J, Pingali SG (1996) A camera-based system for tracking people in real time. Proceedings of the 13th International Conference on Pattern Recognition, 0–8186-7282-X
Sexton G, Zhang X, Redpath D, Greaves D (1995) Advances in automated pedestrian counting, European Convention on Security and Detection, 0–85296–640-7
Shio A, Sklansky J (1991) Segmentation of people in motion, 1991 Proceedings of the IEEE workshop on Visual Motion, 0–8186–2153-2
Skapura DM (1996) Building neural networks. ACM press, New York
Talu MF, Turkoglu I, Cebeci M (2012) A hybrid tracking method for scaled and oriented objects in crowded scenes, Expert systems with applications. 38(11):13 682–13 687
Terada K, Matsubara K (2009) A method for counting multidirection passerby by using circular space-time image. IEEJ Trans EIS 129(6)
Terada K, Yoshida D, Oe S, Yamaguchi J (1999) A method of counting the passing people by using the stereo images. International conference on image processing, 0–7803–5467-2
Terada K, Yoshida D, Oe S, Yamaguchi J (1999) A method of counting the passing people by using the stereo images, International conference on image processing, 0–7803–5467-2
Tesei A, Teschioni A, Regazzoni CS, Vernazza G (1996) Long-Memory" matching of interacting complex objects from real image sequences, 1996 Conference on Time Varying Image Processing and Moving Objects Recognition
Tu PH, Rittscher J, Perera A, Krahnstoever N (2005) Detecting and countingpeople in surveillance applications. In: IEEE International Conference on Advanced Video and Signal Based Surveillance Proceedings of AVSS 2005, p 306–311
Tutorial slides by Andrew Moore (2019). Http://www.cs.cmu.edu/~awm
Vapnik V (1995) The nature of statistical learning theory. Springer, New York ISBN 0–387–94559-8
Vapnik V, Golowich S, Smola A (1997) Support vector method for function approximation, regression estimation, and signal processing. In: Mozer M, Jordan M, Petsche T (eds) Advances in neural information processing systems, vol 9. MIT Press, Cambridge, pp 281–287
Viola P, Jones M, Snow D (2005) Detecting pedestrians using patterns of motion and appearance. Int J Comput Vis 63(2):153–161
Wikipedia Online (2019). Http://en.wikipedia.org/wiki
Wu Y (2008) A new NN-SVM algorithm based on girds [13]. Journal of Chongqing University of Posts and Telecommunications: Natural Science Edition 20(6):706–709
Wu B, Nevatia R (2005) Detection of multiple, partially occluded humans in a single image by Bayesian combination of edgelet part detectors. In: Proc. IEEE Int. Conf. Comput. Vis., vol. 1, p 90–97
Xiao J, Hays J, Ehinger K (2010) Sun database: Large-scale scene recognition from abbey to zoo. Computer Vision and Pattern Recognition (CVPR)
Xu H, Lv P, Meng L (2010) A people counting system based on head-shoulder detection and tracking insurveillance video. In: 2010 International Conference on Computer Design and Applications, p V1394–V1398
Yang Y, Ramanan D (2011) Articulated pose estimation with exible mixtures-of-parts. Computer Vision and Pattern Recognition (CVPR)
Ye W, Xu Y, Zhong Z (2007) Robust people counting in crowded environment. In: 2007 IEEE International Conference on Robotics and Biomimetics, p 1133–1137
Yu HB, Liu L (2008) A vision-based method to estimate passenger flow in bus. Journal of Image and Graphics (15):716–722
Zavaschi THH, Britto AS Jr, Oliveira LES, Koerich AL (2013) Fusion of feature sets and classifiers for facial expression recognition. Expert Syst Appl 40(2):646–655
Zhang E, Chen F (2007) A fast and robust people counting method in video surveillance. In: Proceedings of 2007 International Conference on Computational Intelligence and Security, p 339–343
Zhang X, Gao Y (2009) Face recognition across pose: A review. Pattern Recogn
Zhang D, Guo G, Huang D, Han J PoseFlow: a deep motion representation for understanding human behaviors in videos. CVPR
Zhang D, Han J SPFTN: A Joint Learning Framework for Localizing and Segmenting Objects in Weakly Labeled Videos. IEEE Trans Pattern Anal Mach Intell
Zhao T, Nevatia R (2003) Bayesian human segmentation in crowded situations. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., vol. 2, p 459–466
Zhao T, Nevatia R (2004) Tracking multiple humans in crowded environment. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., p II-406–II-413
Zhao T, Nevatia R, Wu B (2008) Segmentation and tracking of multiple humans in crowded environments. IEEE Trans Pattern AnalMach Intell 30(7):1198–1211
Zheng Q, Yang M (2017) A video stabilization method based on inter-frame image matching score. Glob J Comput Sci Technol 17:41–46
Zhu AZ, Yuan L, Chaney K, Daniilidis K. EV-FlowNet: self-supervised optical flow estimation for event-based cameras, https://arxiv.org/abs/1802.06898
Zulkifley MA, Moran B (2012) Robust hierarchical multiple hypothesis tracker for multiple-object tracking. Expert Syst Appl 39(16):1231912331
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Haq, E.U., Huarong, X., Xuhui, C. et al. A fast hybrid computer vision technique for real-time embedded bus passenger flow calculation through camera. Multimed Tools Appl 79, 1007–1036 (2020). https://doi.org/10.1007/s11042-019-08167-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-019-08167-y