Skip to main content
SpringerLink
Log in

Research on pedestrian counting based on millimeter wave

  • Regular Paper
  • Published:
CCF Transactions on Pervasive Computing and Interaction Aims and scope Submit manuscript

Abstract

The thriving development of big data has provided powerful techniques for pedestrian counting and monitoring in public places, which has attracted great attention. Current pedestrian counting solutions based on computer vision and sensors have privacy leakage issues and cost issues. As a wireless technique, millimeter wave has diverse application scenarios because of its advantages of safety, cheapness and stability. In this paper, we propose a novel system to realize pedestrian counting leveraging millimeter wave. Based on the point cloud data collected by radar, we design an efficient denoising algorithm based on the point cloud distribution and transform the problem into a multi-classification task, and then extract features from pedestrian trajectories to form a series of density maps describing pedestrians’ movement information. We also propose a classifier model mmCountNet to accurately predict the pedestrian and build a data set containing 2000 samples for model training. The experiment results show that the accuracy of the proposed system is 95.92% in the Single-Crowd testing, and 91.35% in the Multi-Crowds testing, which basically achieves the purpose of pedestrian counting.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Algorithm 1
Fig. 13
Algorithm 2
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

Similar content being viewed by others

Data availability

In our work, we designed and collected a dataset containing 2000 samples with a total time of 6.7 h for training the model. If readers need more details about the dataset, they can contact corresponding author through email.

References

  • Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE computer society conference on computer vision and pattern recognition, pp. 886–893 (2005)

  • Dollár, P., Tu, Z., Perona, S. P.: Belongie: integral channel features. In: Proceedings of the British Machine Vision Conference, pp. 1–11 (2009)

  • Feng, Y., Zhang, K., Wang, C., Xie, L., Ning, J., Chen, S.: mmeavesdropper: signal augmentation-based directional eavesdropping with mmwave radar. In: IEEE INFOCOM 2023-IEEE Conference on Computer Communications, pp. 1–10 (2023)

  • Filipic, J., Biagini, M., Mas, I., Pose, C.D., Giribet, J.I., Parisi, D.R.: People counting using visible and infrared images. Neurocomputing 450, 25–32 (2021)

    Article  Google Scholar 

  • Hafeezallah, A., Al-Dhamari, A., Abu-Bakar, S.A.R.: U-asd net: supervised crowd counting based on semantic segmentation and adaptive scenario discovery. IEEE Access. 9, 127444–127459 (2021)

    Article  Google Scholar 

  • Kianoush, S., Savazzi, S., Rampa, V., Nicoli, M.: People counting by dense WiFi MIMO networks: channel features and machine learning algorithms. Sensors. 19(16), 3450 (2019)

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  • Kong, H., Chen, Z., Yue, W., Ni, K.: Improved YOLOv4 for pedestrian detection and counting in UAV images. Comput. Intell. Neurosci. 2022, 6106853 (2022)

    Article  PubMed  PubMed Central  Google Scholar 

  • Lee, S.P., Kini, N.P., Peng, W.H., Ma, C.W., Hwang, J.N.: Hupr: A benchmark for human pose estimation using millimeter wave radar. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 5715–5724 (2023)

  • Lesani, A., Nateghinia, E., Miranda-Moreno, L.F.: Development and evaluation of a real-time pedestrian counting system for high-volume conditions based on 2d lidar. Transp. Res. Part C Emerg. Technol. 114, 20–35 (2020)

    Article  Google Scholar 

  • Li, H., Liu, S. R.and Wang, Jiang, W., Lu, C.X.: Pedestrian liveness detection based on mmWave radar and camera fusion. In: 19th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON), pp. 262–270 (2022)

  • Li, J., He, Q., Zhang, C., Wang, H.: An indoor people counting model based on global attention. Comp. Eng. Sci./Jisuanji Gongcheng yu Kexue. 44(3), 471–478 (2022)

    Google Scholar 

  • Liu, H., Wang, Y., Zhou, A., He, H., Wang, W., Wang, K., Pan, P., Lu, Y., Liu, L., Ma, H.: Real-time arm gesture recognition in smart home scenarios via millimeter wave sensing. Proc. ACM Inter. Mob. Weara. Ubiquitous Technol. 4(4), 1–28 (2020)

    Article  Google Scholar 

  • Liu, L., Jiang, J., Jia, W., Amirgholipour, S., Wang, Y., Zeibots, M., He, X.: Denet: a universal network for counting crowd with varying densities and scales. IEEE Trans. Multimedia 23, 1060–1068 (2020)

    Article  Google Scholar 

  • Lv, M., Zhang, W., Zhang, C., Liu, X.: A pedestrian sensing warning system with edge compute module on millimeter wave radar. In: International Conference on Transportation and Development, pp. 261–270 (2022)

  • Meng, Z., Fu, S., Yan, J., Liang, H., Zhou, A., Zhu, S., Ma, H., Liu, J.: Gait recognition for co-existing multiple people using millimeter wave sensing. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 849–856 (2020)

  • Pham, C.T., Luong, V.S., Nguyen, D.K., Vu, H.H.T., Le, M.: Convolutional neural network for people counting using UWB impulse radar. J. Instrum. 16(08), 08031 (2021)

    Article  Google Scholar 

  • Sandaruwan, R., Alagiyawanna, I., Sandeepa, S., Dias, S., Dias, D.: Device-free pedestrian count estimation using wi-fi channel state information. In: 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pp. 2610–2616 (2021)

  • Shahzad, A.R., Jalal, A.: A smart surveillance system for pedestrian tracking and counting using template matching. In 2021 International Conference on Robotics and Automation in Industry (ICRAI), pp. 1-6 (2021)

  • Shang, H., Sun, L., Qin, W.: Pedestrian detection at night based on the fusion of infrared camera and millimeter-wave radar. Chin. J. Sens. Actuat. 34(9), 1137–1145 (2020)

    Google Scholar 

  • Shen, J., Xiong, X., Xue, Z., Bian, Y.: A convolutional neural-network-based pedestrian counting model for various crowded scenes. Comput.-Aided Civ. Infrastruct. Eng. 34(10), 897–914 (2019)

    Article  Google Scholar 

  • Wang, H., Zhang, D., Ma, J., Wang, Y., Wang, Y., Wu, D., Gu, T., Xie, B.: Human respiration detection with commodity WiFi devices: Do user location and body orientation matter? In: Proceedings of the 2016 ACM international joint conference on pervasive and ubiquitous computing, pp. 25–36 (2016)

  • Xiao, J., Ding, L., Shu, X., Zhang, W.: A computer vision-based pedestrian traffic counting method. Inform. Technol. 39(8), 22–25 (2015)

    Google Scholar 

  • Yan, B., Wang, P., Du, L., Chen, X., Fang, Z., Wu, Y.: mmgesture: Semi-supervised gesture recognition system using mmwave radar. Expert Syst. Appl. 213, 119042 (2023)

    Article  Google Scholar 

  • Yang, Y., Xu, H., Chen, Q., Cao, J., Wang, Y.: Multi-vib: precise multi-point vibration monitoring using mmwave radar. Proc. ACM Inter. Mob. Wearab. Ubiquitous Technol. 6(4), 1–26 (2023)

    CAS  Google Scholar 

  • Yao, Y., Zhang, X., Liang, X.Y., Zhang, S.F., Zhao, J.: A real-time pedestrian counting system based on rgb-d. In: 2020 12th international conference on advanced computational intelligence (ICACI), pp. 110-117 (2020)

  • Yu, Y., Qin, X., Hussain, S., Hou, W., Weis, T.: Pedestrian counting based on piezoelectric vibration sensor. Appl. Sci. 12(4), 1920 (2022)

    Article  CAS  Google Scholar 

  • Zeng, Y., Wu, D., Xiong, J., Yi, E., Gao, R., Zhang, D.: Farsense: Pushing the range limit of wifi-based respiration sensing with csi ratio of two antennas. Proc. ACM Inter. Mob. Wearab. Ubiquitous Technol. 3(3), 1–26 (2019)

    Article  Google Scholar 

  • Zeng, Y., Wu, D., Xiong, J., Liu, J., Liu, Z., Zhang, D.: Multisense: enabling multi-person respiration sensing with commodity wifi. Proc. ACM Inter. Mob. Wearab. Ubiquitous Technol. 4(3), 1–29 (2020)

    Article  Google Scholar 

  • Zeng, S., Wan, H., Shi, S., Wang, W.: msilent: towards general corpus silent speech recognition using cots mmwave radar. Proc. ACM Inter. Mob. Wearab. Ubiquitous Technol. 7(1), 1–28 (2023)

    Google Scholar 

  • Zhang, Z., Wang, X., Huang, D., Fang, X., Zhou, M., Zhang, Y.: Mrpt: millimeter-wave radar-based pedestrian trajectory tracking for autonomous urban driving. IEEE Trans. Instrum. Meas. 71, 1–17 (2021)

    Google Scholar 

  • Zhao, P., Lu, C.X., Wang, J., Chen, C., Wang, W., Trigoni, N., Markham, A.: Human tracking and identification through a millimeter wave radar. Ad Hoc Netw. 116, 102475 (2021)

    Article  Google Scholar 

  • Zhou, W., Pan, Y., Lei, J., Ye, L., Yu, L.: Defnet: dual-branch enhanced feature fusion network for rgb-t crowd counting. IEEE Trans. Intell. Transp. Syst. 23(12), 24540–24549 (2022)

    Article  Google Scholar 

  • Zhu, J., Cao, J., Yang, Y., Ren, W., Han, H.: mmDrive: Fine-grained fatigue driving detection using mmWave radar. ACM Trans. Internet Things 4(4), 1–30 (2023)

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported in part by National Key R &D Program of China under Grant No. 2022YFB3303900; National Natural Science Foundation of China under Grant Nos. 62372224, 62272216, 61832008; The Key R &D Program of Jiangsu Province under Grant BE2020001–3. This work is partially supported by Collaborative Innovation Center of Novel Software Technology and Industrialization under Grant Nos. 62272216. This work is partially supported by the Fundamental Research Funds for the Central Universities No. 2023300175(020214380100). Chuyu Wang is the corresponding author.

Author information

Authors and Affiliations

  1. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210023, China

    Jiayang Zhao, Chuyu Wang, Lei Xie, Yiwen Feng & Sanglu Lu

Authors
  1. Jiayang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chuyu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lei Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yiwen Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sanglu Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chuyu Wang.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, J., Wang, C., Xie, L. et al. Research on pedestrian counting based on millimeter wave. CCF Trans. Pervasive Comp. Interact. 6, 82–100 (2024). https://doi.org/10.1007/s42486-023-00145-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42486-023-00145-6

Keywords

Search

Navigation