Skip to main content

Advertisement

Springer Nature Link
Log in

Exploring density rectification and domain adaption method for crowd counting

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Crowd counting has received increasing attention due to its important roles in multiple fields, such as social security, commercial applications, epidemic prevention and control. To this end, we explore two critical issues that seriously affect the performance of crowd counting including nonuniform crowd density distribution and cross-domain problems. Aiming at the nonuniform crowd density distribution issue, we propose a density rectifying network (DRNet) that consists of several dual-layer pyramid fusion modules (DPFM) and a density rectification map (DRmap) auxiliary learning module. The proposed DPFM is embedded into DRNet to integrate multi-scale crowd density features through dual-layer pyramid fusion. The devised DRmap auxiliary learning module further rectifies the incorrect crowd density estimation by adaptively weighting the initial crowd density maps. With respect to the cross-domain issue, we develop a domain adaptation method of randomly cutting mixed dual-domain images, which learns domain-invariance features and decreases the domain gap between the source domain and the target domain from global and local perspectives. Experimental results indicate that the devised DRNet achieves the best mean absolute error (MAE) and competitive mean squared error (MSE) compared with other excellent methods on four benchmark datasets. Additionally, a series of cross-domain experiments are conducted to demonstrate the effectiveness of the proposed domain adaption method. Significantly, when the A and B parts of the Shanghaitech dataset are the source domain and target domain respectively, the proposed domain adaption method decreases the MAE of DRNet by \(47.6\%\).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Availability of data and materials

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  2. Wang Q, Gao J, Lin W, Yuan Y (2019) Learning from synthetic data for crowd counting in the wild. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8198–8207

  3. Wang Q, Han T, Gao J, Yuan Y (2021) Neuron linear transformation: modeling the domain shift for crowd counting. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2021.3051

    Article  Google Scholar 

  4. Yang Y, Li G, Wu Z, Su L, Huang Q, Sebe N (2020) Reverse perspective network for perspective-aware object counting. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4374–4383 (2020)

  5. Wang M, Cai H, Zhou J, Gong M (2021) Interlayer and intralayer scale aggregation for scale-invariant crowd counting. Neurocomputing 441:128–137

    Article  Google Scholar 

  6. Peng S, Wang L, Yin B, Li Y, Xia Y, Hao X (2021) Adaptive weighted crowd receptive field network for crowd counting. Pattern Anal Appl 24(2):805–817

    Article  Google Scholar 

  7. Sam DB, Sajjan NN, Maurya H, Babu RV (2019) Almost unsupervised learning for dense crowd counting. In: Proceedings of the AAAI conference on artificial intelligence, pp 8868–8875

  8. Sindagi VA, Yasarla R, Babu DS, Babu RV, Patel VM (2020) Learning to count in the crowd from limited labeled data. In: Proceedings of the european conference on computer vision, pp 212–229

  9. Hu Y, Jiang X, Liu X, Zhang B, Han J, Cao X, Doermann D (2020) Nas-count: counting-by-density with neural architecture search. In: Proceedings of the european conference on computer vision, pp 747–766

  10. Liu J, Gao C, Meng D, Hauptmann AG (2018) Decidenet: counting varying density crowds through attention guided detection and density estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5197–5206 (2018)

  11. Gao J, Wang Q, Yuan Y (2019) Scar:spatial-/channel-wise attention regression networks for crowd counting. Neurocomputing 363:1–8

    Article  Google Scholar 

  12. Gao J, Yuan Y, Wang Q (2021) Feature-aware adaptation and density alignment for crowd counting in video surveillance. IEEE Trans Cybernetics 51(10):4822–4833

    Article  Google Scholar 

  13. Amirgholipour, S., He, X., Jia, W., Wang, D., Zeibots M (2018) A-CCNN: adaptive CCNN for density estimation and crowd counting. In: Proceedings of the IEEE international conference on image processing, pp 948–952. IEEE

  14. Liu W, Salzmann M, Fua P (2019) Context-aware crowd counting. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5099–5108

  15. Zhang Y, Zhou D, Chen S, Gao S, Ma Y (2016) Single-image crowd counting via multi-column convolutional neural network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 589–597

  16. Babu Sam D, Surya S, Venkatesh Babu R (2017) Switching convolutional neural network for crowd counting. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5744–5752

  17. Sindagi VA, Patel VM (2017) Generating high-quality crowd density maps using contextual pyramid cnns. In: Proceedings of the IEEE international conference on computer vision, pp 1861–1870

  18. Cheng Z-Q, Li J-X, Dai Q, Wu X, He J-Y, Hauptmann AG (2019) Improving the learning of multi-column convolutional neural network for crowd counting. In: Proceedings of the 27th ACM international conference on multimedia, pp 1897–1906

  19. Sam DB, Babu RV (2018) Top-down feedback for crowd counting convolutional neural network. In: Proceedings of the AAAI conference on artificial intelligence, pp 7323–7330

  20. Jiang X, Xiao Z, Zhang B, Zhen X, Cao X, Doermann D, Shao L (2019) Crowd counting and density estimation by trellis encoder-decoder networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6133–6142

  21. Liu Y, Wen Q, Chen H, Liu W, Qin J, Han G, He S (2020) Crowd counting via cross-stage refinement networks. IEEE Trans Image Process 29:6800–6812

    Article  MATH  Google Scholar 

  22. Liu X, Van De Weijer J, Bagdanov AD (2019) Exploiting unlabeled data in cnns by self-supervised learning to rank. IEEE Trans Pattern Anal Machine Intell 41(8):1862–1878

    Article  Google Scholar 

  23. Gao J, Wang Q, Li X (2019) Pcc net: perspective crowd counting via spatial convolutional network. IEEE Trans Circuits Syst Video Technol 30(10):3486–3498

    Article  Google Scholar 

  24. Shi Z, Zhang L, Sun Y, Ye Y (2018) Multiscale multitask deep netvlad for crowd counting. IEEE Trans Industrial Inform 14(11):4953–4962

    Article  Google Scholar 

  25. Zhao M, Zhang J, Zhang C, Zhang W (2019) Leveraging heterogeneous auxiliary tasks to assist crowd counting. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 12736–12745

  26. Jiang X, Zhang L, Zhang T, Lv P, Zhou B, Pang Y, Xu M, Xu C (2020) Density-aware multi-task learning for crowd counting. IEEE Trans Multimed 23:443–453

    Article  Google Scholar 

  27. Zhang Q, Chan AB (2019) Wide-area crowd counting via ground-plane density maps and multi-view fusion cnns. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8297–8306

  28. Zhang Q, Lin W, Chan AB (2021) Cross-view cross-scene multi-view crowd counting. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 557–567

  29. Peng T, Li Q, Zhu P (2020) Rgb-t crowd counting from drone: a benchmark and mmccn network. In: Proceedings of the Asian conference on computer vision, pp 497–513

  30. Wen L, Du D, Zhu P, Hu Q, Wang Q, Bo L, Lyu S (2021) Detection, tracking, and counting meets drones in crowds: a benchmark. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7812–7821

  31. Bai Z, Wang Z, Wang J, Hu D, Ding E (2021) Unsupervised multi-source domain adaptation for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 12914–12923

  32. Faraki M, Yu X, Tsai Y-H, Suh Y, Chandraker M (2021) Cross-domain similarity learning for face recognition in unseen domains. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 15292–15301

  33. Fu Y, Zhang M, Xu X, Cao Z, Ma C, Ji Y, Zuo K, Lu H (2021) Partial feature selection and alignment for multi-source domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 16654–16663

  34. He J, Jia X, Chen S, Liu J (2021) Multi-source domain adaptation with collaborative learning for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 11008–11017

  35. Zhang C, Li H, Wang X, Yang X (2015) Cross-scene crowd counting via deep convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 833–841

  36. Hossain MA, Kumar M, Hosseinzadeh M, Chanda O, Wang Y (2019) One-shot scene-specific crowd counting. In: Proceedings of the British machine vision conference, pp 1–11

  37. Li W, Yongbo L, Xiangyang X (2019) Coda: Counting objects via scale-aware adversarial density adaption. In: Proceedings of the International conference on multimedia and expo, pp 193–198

  38. Han T, Gao J, Yuan Y, Wang Q (2020) Focus on semantic consistency for cross-domain crowd understanding. In: ICASSP 2020-2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 1848–1852 . IEEE

  39. He Y, Ma Z, Wei X, Hong X, Ke W, Gong Y (2021) Error-aware density isomorphism reconstruction for unsupervised cross-domain crowd counting. In: Proceedings of the AAAI conference on artificial intelligence, pp 1540–1548

  40. Liu Y, Xu D, Ren S, Wu H, Cai H, He S (2021) Fine-grained domain adaptive crowd counting via point-derived segmentation. arXiv preprint arXiv:2108.02980

  41. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  42. Sandwell DT (1987) Biharmonic spline interpolation of geos-3 and seasat altimeter data. Geophys Res Lett 14(2):139–142

    Article  Google Scholar 

  43. Lin T-Y, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2117–2125

  44. Yun S, Han D, Oh SJ, Chun S, Choe J, Yoo Y (2019) Cutmix: Regularization strategy to train strong classifiers with localizable features. In: Proceedings of the IEEE international conference on computer vision, pp 6023–6032

  45. Peng S, Yin B, Hao X, Yang Q, Kumar A, Wang L (2021) Depth and edge auxiliary learning for still image crowd density estimation. Pattern Anal Appl 24(4):1777–1792

    Article  Google Scholar 

  46. Idrees H, Tayyab M, Athrey K, Zhang D, Al-Maadeed S, Rajpoot N, Shah M (2018) Composition loss for counting, density map estimation and localization in dense crowds. In: Proceedings of the European conference on computer vision, pp 532–546

  47. Sindagi V, Yasarla R, Patel VM (2022) Jhu-crowd++: Large-scale crowd counting dataset and a benchmark method. IEEE Trans Pattern Anal Machine Intell 44(5):2594–2609

    Google Scholar 

  48. Wang Q, Gao J, Lin W, Li X (2020) Nwpu-crowd: a large-scale benchmark for crowd counting and localization. IEEE Trans Pattern Anal Machine intell 43(6):2141–2149

    Article  Google Scholar 

  49. Sam DB, Sajjan NN, Babu RV, Srinivasan M (2018) Divide and grow: capturing huge diversity in crowd images with incrementally growing cnn. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3618–3626

  50. Liu N, Long Y, Zou C, Niu Q, Pan L, Wu H (2019) Adcrowdnet: an attention-injective deformable convolutional network for crowd understanding. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3225–3234

  51. Ma Z, Wei X, Hong X, Gong Y (2019) Bayesian loss for crowd count estimation with point supervision. In: Proceedings of the IEEE International conference on computer vision, pp 6142–6151

  52. Xiong H, Lu H, Liu C, Liu L, Cao Z, Shen C (2019) From open set to closed set: counting objects by spatial divide-and-conquer. In: Proceedings of the IEEE international conference on computer vision, pp 8362–8371

  53. Xu C, Qiu K, Fu J, Bai S, Xu Y, Bai X (2019) Learn to scale: generating multipolar normalized density maps for crowd counting. In: Proceedings of the IEEE international conference on computer vision, pp 8382–8390

  54. Yan Z, Yuan Y, Zuo W, Tan X, Wang Y, Wen S, Ding E (2019) Perspective-guided convolution networks for crowd counting. In: Proceedings of the IEEE international conference on computer vision, pp 952–961

  55. Liu X, Yang J, Ding W, Wang T, Wang Z, Xiong J (2020) Adaptive mixture regression network with local counting map for crowd counting. In: Proceedings of the European conference on computer vision, pp 241–257

  56. Jiang X, Zhang L, Xu M, Zhang T, Lv P, Zhou B, Yang X, Pang Y (2020) Attention scaling for crowd counting. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 4706–4715

  57. Miao Y, Lin Z, Ding G, Han J (2020) Shallow feature based dense attention network for crowd counting. In: Proceedings of the AAAI conference on artificial intelligence, pp 11765–11772

  58. Oh M-h, Olsen P, Ramamurthy KN (2020) Crowd counting with decomposed uncertainty. In: Proceedings of the AAAI conference on artificial intelligence, pp 11799–11806

  59. Wan J, Liu Z, Chan AB (2021) A generalized loss function for crowd counting and localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1974–1983

  60. Zhang S, Zhang X, Li H, He H, Song D, Wang L (2022) Hierarchical pyramid attentive network with spatial separable convolution for crowd counting. Eng Appl Artif Intell 108:1–10

    Article  Google Scholar 

  61. Yan L, Zhang L, Zheng X, Li F (2022) Deeper multi-column dilated convolutional network for congested crowd understanding. Neural Comput Appl 34(2):1407–1422

    Article  Google Scholar 

  62. Sindagi VA, Patel VM (2017) Cnn-based cascaded multi-task learning of high-level prior and density estimation for crowd counting. In: Proceedings of the IEEE international conference on advanced video and signal based surveillance, pp 1–6

  63. Li Y, Zhang X, Chen D (2018) Csrnet: Dilated convolutional neural networks for understanding the highly congested scenes. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1091–1100

  64. Cao X, Wang Z, Zhao Y, Su F (2018) Scale aggregation network for accurate and efficient crowd counting. In: Proceedings of the European conference on computer vision, pp 734–750

  65. Liu L, Qiu Z, Li G, Liu S, Ouyang W, Lin L (2019) Crowd counting with deep structured scale integration network. In: Proceedings of the IEEE international conference on computer vision, pp 1774–1783

  66. Sindagi VA, Patel VM (2019) Multi-level bottom-top and top-bottom feature fusion for crowd counting. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 1002–1012

  67. Gao J, Lin W, Zhao B, Wang D, Gao C, Wen J (2019) C^3 framework: An open-source pytorch code for crowd counting. arXiv preprint arXiv:1907.02724

  68. Shi Z, Zhang L, Liu Y, Cao X, Ye Y, Cheng MM, Zheng G (2018) Crowd counting with deep negative correlation learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5382–5390

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China under grant No.62133013 and sponsored by the CAAI-Huawei MindSpore Open Fund.

Author information

Authors and Affiliations

  1. Department of Automation, University of Science and Technology of China, Huangshan Road, Hefei, 230027, Anhui, China

    Sifan Peng, Baoqun Yin, Qianqian Yang, Qing He & Luyang Wang

Authors
  1. Sifan Peng
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Baoqun Yin
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Qianqian Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Qing He
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Luyang Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Baoqun Yin.

Ethics declarations

Conflict of interest

The authors declared that they have no conflicts of interest in this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peng, S., Yin, B., Yang, Q. et al. Exploring density rectification and domain adaption method for crowd counting. Neural Comput & Applic 35, 3551–3569 (2023). https://doi.org/10.1007/s00521-022-07917-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07917-8

Keywords