Skip to main content
SpringerLink
Log in

Appearance-based passenger counting in cluttered scenes with lateral movement compensation

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Autonomous passenger counting in public transportation represents an integral part of an intelligent transportation system, as it provides vital information to improve the efficiency and resource management of a public transportation network. However, counting passengers in highly crowded scenes is a challenging task due to their random movement, diverse appearance settings and inter-object occlusions. Furthermore, state-of-the-art methods in this domain rely heavily on additional custom cameras or sensors instead of existing onboard surveillance cameras, which consequently limits the feasibility of such systems for large-scale deployment. Hence, this paper puts forward an enhanced appearance descriptor with lateral movement compensation, which addresses the difficulty in counting passengers bidirectionally in cluttered scenes. We first construct a head re-identification dataset, which is used to train an appearance descriptor. This dataset addresses the absence of a person re-identification dataset, which in turn allows for accurate tracking of passengers in cluttered scenes. Then, a novel technique of applying a fedora counting line is introduced to count the number of passengers entering and exiting a bus. This technique compensates the impact of passengers’ lateral movement, which crucially increases the accuracy of bidirectional passenger counting using onboard bus surveillance cameras. In addition, a real-time implementation of the proposed method, which includes the integration of DeepStream and fedora counting line, is also presented. Experimental results on a challenging test dataset demonstrate that the proposed method outperforms benchmarked techniques with an average counting accuracy of 93.21% for entering and 96.10% for exiting public buses. Furthermore, the proposed system achieves this accuracy at an average frame rate of 16 frames per second, which represents a practical solution to a real-time application.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

Notes

  1. https://github.com/rickysutopo/HRID.

References

  1. Advancing Public Transport (2015) Video Surveillance in Public Transport. Advancing Public Transport Portal Official

  2. An L, Chen X, Yang S, Bhanu B (2016) Sparse representation matching for person re-identification. Inf Sci 355–356:74–89. https://doi.org/10.1016/j.ins.2016.02.055

    Article  Google Scholar 

  3. Cho SI, Kang SJ (2018) Real-time people counting system for customer movement analysis. IEEE Access 6:55264–55272. https://doi.org/10.1109/ACCESS.2018.2872684

    Article  Google Scholar 

  4. Greer L, Fraser JL, Hicks D, Mercer M, Thompson K et al (2018) Intelligent transportation systems benefits, costs, and lessons learned: 2018 update report. Technical report, United States. Department of Transportation. ITS Joint Program Office

  5. Hirzer M, Beleznai C, Roth PM, Bischof H (2011) Person re-identification by descriptive and discriminative classification. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 6688 LNCS, pp 91–102. https://doi.org/10.1007/978-3-642-21227-7_9

  6. Israel GD (1992) Determining sample size

  7. Kaiser J, Friedrich A, Tieck JCV, Reichard D, Roennau A, Neftci E, Dillmann R (2019) Embodied neuromorphic vision with event-driven random backpropagation, pp 1–8

  8. Li F, Yang FW, Liang HW, Yang WM (2017) Automatic passenger counting system for bus based on RGB-D video 117:209–220. https://doi.org/10.2991/eeeis-16.2017.29

    Article  Google Scholar 

  9. Li H, Xu J, Zhu J, Tao D, Yu Z (2019) Top distance regularized projection and dictionary learning for person re-identification. Inf Sci 502:472–491. https://doi.org/10.1016/j.ins.2019.06.046

    Article  Google Scholar 

  10. Li W, Zhao R, Xiao T, Wang X (2014) DeepReID: Deep filter pairing neural network for person re-identification. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 152–159. https://doi.org/10.1109/CVPR.2014.27

  11. Liciotti D, Cenci A, Frontoni E, Mancini A, Zingaretti P (2017) An intelligent RGB-D video system for bus passenger counting, pp 473–484. https://doi.org/10.1007/978-3-319-48036-7_34

  12. Liu G, Yin Z, Jia Y, Xie Y (2017) Passenger flow estimation based on convolutional neural network in public transportation system. Knowl-Based Syst 123:102–115. https://doi.org/10.1016/j.knosys.2017.02.016

    Article  Google Scholar 

  13. Nakashima H, Arai I, Fujikawa K (2019) Passenger counter based on random forest regressor using drive recorder and sensors in buses. In: 2019 IEEE international conference on pervasive computing and communications workshops, PerCom workshops, pp 561–566. https://doi.org/10.1109/PERCOMW.2019.8730761

  14. Nalepa J, Szymanek J, Kawulok M (2015) Real-time people counting from depth images. In: Kozielski S, Mrozek D, Kasprowski P, Małysiak-Mrozek B, Kostrzewa D (eds) Communications in computer and information science, communications in computer and information science, vol 521, pp 387–397. Springer, Cham. https://doi.org/10.1007/978-3-319-18422-7_34

  15. NV5 Engineering: Bus Stop Design and Safety Guideline Handbook. Technical report (2014)

  16. Perng JW, Wang TY, Hsu YW, Wu BF (2016) The design and implementation of a vision-based people counting system in buses. In: 2016 IEEE international conference on system science and engineering, ICSSE 2016 pp 1–3 . https://doi.org/10.1109/ICSSE.2016.7551620

  17. Redmon J, Farhadi A (2018) YOLOv3: an incremental improvement

  18. Sun S, Akhtar N, Song H, Zhang C, Li J, Mian A (2019) Benchmark data and method for real-time people counting in cluttered scenes using depth sensors. IEEE Trans Intell Transp Syst 20(10):3599–3612. https://doi.org/10.1109/tits.2019.2911128

    Article  Google Scholar 

  19. Tang S, Ye M, Xu P, Li X (2019) Adaptive pedestrian detection by predicting classifier. Neural Comput Appl 31(4):1189–1200. https://doi.org/10.1007/s00521-017-3152-z

    Article  Google Scholar 

  20. R. Service Transportation Unit (2005) Bus stop design guide bus stop design guide 1. Technical report

  21. Wang B, Cao G, Shang Y, Zhou L, Zhang Y, Li X (2020) Single-column CNN for crowd counting with pixel-wise attention mechanism. Neural Comput Appl 32(7):2897–2908. https://doi.org/10.1007/s00521-018-3810-9

    Article  Google Scholar 

  22. Wojke N, Bewley A, Paulus D (2017) Simple online and realtime tracking with a deep association metric. In: 2017 IEEE international conference on image processing (ICIP), pp 3645–3649. IEEE. https://doi.org/10.1109/ICIP.2017.8296962

  23. Yahiaoui T, Khoudour L, Meurie C (2010) Real-time passenger counting in buses using dense stereovision. J Electron Imaging 19(3):031202

    Article  Google Scholar 

  24. Yang B, Cao J, Liu X, Wang N, Lv J (2018) Edge computing-based real-time passenger counting using a compact convolutional neural network. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3894-2

  25. Zhang X, Jing XY, Zhu X, Ma F (2020) Semi-supervised person re-identification by similarity-embedded cycle GANs. Neural Comput Appl. https://doi.org/10.1007/s00521-020-04809-7

    Article  Google Scholar 

  26. Zheng L, Bie Z, Sun Y, Wang J, Su C, Wang S, Tian Q (2016) MARS: a video benchmark for large-scale person re-identification. In: Lecture Notes in Computer Science (including subseries. Lecture notes in artificial intelligence and lecture notes in bioinformatics), Lecture notes in computer science, vol 9910, pp 868–884. Springer, Cham. https://doi.org/10.1007/978-3-319-46466-4_52

  27. Zheng L, Shen L, Tian L, Wang S, Wang J, Tian Q (2015) Scalable person re-identification: a Benchmark University of Texas at San Antonio. ICCV, pp 1116–1124. https://doi.org/10.1109/ICCV.2015.133

Download references

Acknowledgements

This work was supported by the School of Engineering and School of Information Technology, Monash University Malaysia, by Intel Technology Sdn Bhd, by Grants from the MSCA-Rise European Project IDENTITY, the Italian Ministry Research projects PRIN COSMOS and SPADA, and by a special Grant from the University of Sassari “fondo di Ateneo per la Ricerca 2019 e 2020”.

Author information

Authors and Affiliations

  1. School of Engineering, Monash University Malaysia, Jalan Lagoon Selatan, 47500, Bandar Sunway, Selangor, Malaysia

    Ricky Sutopo & Joanne Mun-Yee Lim

  2. School of Information Technology, Monash University Malaysia, Jalan Lagoon Selatan, 47500, Bandar Sunway, Selangor, Malaysia

    Vishnu Monn Baskaran & KokSheik Wong

  3. Computer Vision Laboratory, Department of Agriculture and Information Engineering, University of Sassari, Sassari, 07100, Italy

    Massimo Tistarelli

  4. Intel Corporation, Jalan Sultan Azlan Shah, Kawasan Perindustrian Bayan Lepas, 11900, Bayan Lepas, Pulau Pinang, Malaysia

    Heng Fui Liau

Authors
  1. Ricky Sutopo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joanne Mun-Yee Lim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vishnu Monn Baskaran
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. KokSheik Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Massimo Tistarelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Heng Fui Liau
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joanne Mun-Yee Lim.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sutopo, R., Lim, J.MY., Baskaran, V.M. et al. Appearance-based passenger counting in cluttered scenes with lateral movement compensation. Neural Comput & Applic 33, 9891–9912 (2021). https://doi.org/10.1007/s00521-021-05760-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-05760-x

Keywords

Search

Navigation