skip to main content
10.1145/3571600.3571652acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicvgipConference Proceedingsconference-collections
research-article

Dense or Sparse: Crowd Counting with Binary Supervision

Published: 12 May 2023 Publication History

Abstract

Dense crowd counting is one of the challenging problems where creating large labeled datasets turns out to be difficult. Typical crowd images have thousands of people positioned close to each other and annotating the locations of every person is tedious. Add to these the growing need to include crowds from as many diverse scenarios as possible for better generalization. In this context, labeling every head for various settings under consideration is not scalable and directly affects the performance of deep models on account of limited data. We mitigate this issue with a new binary labeling scheme. Every image is simply labeled to either dense or sparse crowd category, instead of annotating every single person in the scene. This leads to dramatic reduction in the amount of annotations required and becomes proportional to the number of images rather than the crowd count. For training counting models, we create noisy density maps directly from the edge density of the images, which are then improved through rectifier networks. There are separate rectifier networks for dense and sparse categories, trained in an unsupervised fashion. The proposed counting model is composed of a self-supervised backbone feature network and a regressor head. The ground truth density maps are generated using the binary labels and the rectifier networks for training the regressor. Experiments show that the proposed architecture achieves competitive performance than existing models at an extremely low annotation cost.

References

[1]
Deepak Babu Sam and R Venkatesh Babu. 2018. Top-Down Feedback for Crowd Counting Convolutional Neural Network. In Proceedings of the AAAI Conference on Artificial Intelligence.
[2]
Deepak Babu Sam, Skand Vishwanath Peri, Mukuntha Narayanan Sundararaman, and R. Venkatesh Babu. 2020. Going Beyond the Regression Paradigm with Accurate Dot Prediction for Dense Crowds. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV).
[3]
Deepak Babu Sam, Skand Vishwanath Peri, Mukuntha Narayanan Sundararaman, Amogh Kamath, and R. Venkatesh Babu. 2020. Locate, Size and Count: Accurately Resolving People in Dense Crowds via Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) (2020).
[4]
Deepak Babu Sam, Neeraj N. Sajjan, R. Venkatesh Babu, and Mukundhan Srinivasan. 2018. Divide and Grow: Capturing Huge Diversity in Crowd Images With Incrementally Growing CNN. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[5]
Deepak Babu Sam, Neeraj N. Sajjan, Himanshu Maurya, and R. Venkatesh Babu. 2019. Almost Unsupervised Learning for Dense Crowd Counting. In Proceedings of the AAAI Conference on Artificial Intelligence.
[6]
Deepak Babu Sam, Shiv Surya, and R Venkatesh Babu. 2017. Switching convolutional neural network for crowd counting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[7]
Shuai Bai, Zhiqun He, Yu Qiao, Hanzhe Hu, Wei Wu, and Junjie Yan. 2020. Adaptive Dilated Network With Self-Correction Supervision for Counting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4594–4603.
[8]
John Canny. 1986. A computational approach to edge detection. IEEE Transactions on pattern analysis and machine intelligence (TPAMI) (1986).
[9]
Zhi-Qi Cheng, Jun-Xiu Li, Qi Dai, Xiao Wu, and Alexander G Hauptmann. 2019. Learning spatial awareness to improve crowd counting. In Proceedings of the IEEE International Conference on Computer Vision (ICCV).
[10]
Yutao Hu, Xiaolong Jiang, Xuhui Liu, Baochang Zhang, Jungong Han, Xianbin Cao, and David Doermann. 2020. NAS-Count: Counting-by-Density with Neural Architecture Search. arXiv preprint arXiv:2003.00217(2020).
[11]
Haroon Idrees, Imran Saleemi, Cody Seibert, and Mubarak Shah. 2013. Multi-source multi-scale counting in extremely dense crowd images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[12]
Haroon Idrees, Muhmmad Tayyab, Kishan Athrey, Dong Zhang, Somaya Al-Maadeed, Nasir Rajpoot, and Mubarak Shah. 2018. Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowds. In Proceedings of the European Conference on Computer Vision (ECCV).
[13]
Xiaolong Jiang, Zehao Xiao, Baochang Zhang, Xiantong Zhen, Xianbin Cao, David Doermann, and Ling Shao. 2019. Crowd counting and density estimation by trellis encoder-decoder networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[14]
Xiaoheng Jiang, Li Zhang, Mingliang Xu, Tianzhu Zhang, Pei Lv, Bing Zhou, Xin Yang, and Yanwei Pang. 2020. Attention Scaling for Crowd Counting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4706–4715.
[15]
Alexander Kolesnikov, Xiaohua Zhai, and Lucas Beyer. 2019. Revisiting self-supervised visual representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (CVPR).
[16]
Yuhong Li, Xiaofan Zhang, and Deming Chen. 2018. CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[17]
Dongze Lian, Jing Li, Jia Zheng, Weixin Luo, and Shenghua Gao. 2019. Density map regression guided detection network for rgb-d crowd counting and localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[18]
Chenchen Liu, Xinyu Weng, and Yadong Mu. 2019. Recurrent attentive zooming for joint crowd counting and precise localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[19]
Jiang Liu, Chenqiang Gao, Deyu Meng, and Alexander G. Hauptmann. 2018. DecideNet: Counting Varying Density Crowds Through Attention Guided Detection and Density Estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[20]
Liang Liu, Hao Lu, Hongwei Zou, Haipeng Xiong, Zhiguo Cao, and Chunhua Shen. 2020. Weighing Counts: Sequential Crowd Counting by Reinforcement Learning. arXiv preprint arXiv:2007.08260(2020).
[21]
Ning Liu, Yongchao Long, Changqing Zou, Qun Niu, Li Pan, and Hefeng Wu. 2019. ADCrowdNet: An attention-injective deformable convolutional network for crowd understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[22]
Weizhe Liu, Mathieu Salzmann, and Pascal Fua. 2019. Context-aware crowd counting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[23]
X. Liu, J. Van De Weijer, and A. D. Bagdanov. 2019. Exploiting Unlabeled Data in CNNs by Self-supervised Learning to Rank. IEEE Transactions on Pattern Analysis and Machine Intelligence.
[24]
Xiyang Liu, Jie Yang, and Wenrui Ding. 2020. Adaptive Mixture Regression Network with Local Counting Map for Crowd Counting. arXiv preprint arXiv:2005.05776(2020).
[25]
Yan Liu, Lingqiao Liu, Peng Wang, Pingping Zhang, and Yinjie Lei. 2020. Semi-Supervised Crowd Counting via Self-Training on Surrogate Tasks. arXiv preprint arXiv:2007.03207(2020).
[26]
Yuting Liu, Miaojing Shi, Qijun Zhao, and Xiaofang Wang. 2019. Point in, box out: Beyond counting persons in crowds. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[27]
Zhiheng Ma, Xing Wei, Xiaopeng Hong, and Yihong Gong. 2019. Bayesian loss for crowd count estimation with point supervision. In Proceedings of the IEEE International Conference on Computer Vision (ICCV).
[28]
Viresh Ranjan, Hieu Le, and Minh Hoai. 2018. Iterative Crowd Counting. In Proceedings of the European Conference on Computer Vision.
[29]
Zan Shen, Yi Xu, Bingbing Ni, Minsi Wang, Jianguo Hu, and Xiaokang Yang. 2018. Crowd Counting via Adversarial Cross-Scale Consistency Pursuit. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[30]
Zenglin Shi, Pascal Mettes, and Cees GM Snoek. 2019. Counting with focus for free. In Proceedings of the IEEE International Conference on Computer Vision (ICCV).
[31]
Zenglin Shi, Le Zhang, Yun Liu, Xiaofeng Cao, Yangdong Ye, Ming-Ming Cheng, and Guoyan Zheng. 2018. Crowd Counting With Deep Negative Correlation Learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[32]
Vishwanath A Sindagi and Vishal M Patel. 2017. CNN-based cascaded multi-task learning of high-level prior and density estimation for crowd counting. In Proceedings of the IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).
[33]
Vishwanath A Sindagi and Vishal M Patel. 2017. Generating high-quality crowd density maps using contextual pyramid CNNs. In Proceedings of the IEEE International Conference on Computer Vision (ICCV).
[34]
Vishwanath A Sindagi and Vishal M Patel. 2019. Multi-level bottom-top and top-bottom feature fusion for crowd counting. In Proceedings of the IEEE International Conference on Computer Vision (ICCV).
[35]
Vishwanath A Sindagi, Rajeev Yasarla, Deepak Sam Babu, R Venkatesh Babu, and Vishal M Patel. 2020. Learning to count in the crowd from limited labeled data. arXiv preprint arXiv:2007.03195(2020).
[36]
Vishwanath A Sindagi, Rajeev Yasarla, and Vishal M Patel. 2019. Pushing the frontiers of unconstrained crowd counting: New dataset and benchmark method. In Proceedings of the IEEE International Conference on Computer Vision (ICCV).
[37]
Vishwanath A Sindagi, Rajeev Yasarla, and Vishal M Patel. 2020. JHU-CROWD++: Large-Scale Crowd Counting Dataset and A Benchmark Method. Technical Report (2020).
[38]
Pongpisit Thanasutives, Ken-ichi Fukui, Masayuki Numao, and Boonserm Kijsirikul. 2020. Encoder-Decoder Based Convolutional Neural Networks with Multi-Scale-Aware Modules for Crowd Counting. arXiv preprint arXiv:2003.05586(2020).
[39]
Yukun Tian, Yiming Lei, Junping Zhang, and James Z Wang. 2019. Padnet: Pan-density crowd counting. IEEE Transactions on Image Processing 29 (2019), 2714–2727.
[40]
Jia Wan, Wenhan Luo, Baoyuan Wu, Antoni B Chan, and Wei Liu. 2019. Residual regression with semantic prior for crowd counting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[41]
Qi Wang, Junyu Gao, Wei Lin, and Yuan Yuan. 2019. Learning from synthetic data for crowd counting in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[42]
Haipeng Xiong, Hao Lu, Chengxin Liu, Liang Liu, Zhiguo Cao, and Chunhua Shen. 2019. From open set to closed set: Counting objects by spatial divide-and-conquer. In Proceedings of the IEEE International Conference on Computer Vision (ICCV).
[43]
Zhaoyi Yan, Yuchen Yuan, Wangmeng Zuo, Xiao Tan, Yezhen Wang, Shilei Wen, and Errui Ding. 2019. Perspective-guided convolution networks for crowd counting. In Proceedings of the IEEE International Conference on Computer Vision (ICCV).
[44]
Yifan Yang, Guorong Li, Zhe Wu, Li Su, Qingming Huang, and Nicu Sebe. 2020. Reverse Perspective Network for Perspective-Aware Object Counting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4374–4383.
[45]
Yifan Yang, Guorong Li, Zhe Wu, Li Su, Qingming Huang, and Nicu Sebe. 2020. Weakly-supervised crowd counting learns from sorting rather than locations. In European Conference on Computer Vision. Springer, 1–17.
[46]
Yingying Zhang, Desen Zhou, Siqin Chen, Shenghua Gao, and Yi Ma. 2016. Single-image crowd counting via multi-column convolutional neural network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[47]
Muming Zhao, Jian Zhang, Chongyang Zhang, and Wenjun Zhang. 2019. Leveraging heterogeneous auxiliary tasks to assist crowd counting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[48]
Zhen Zhao, Miaojing Shi, Xiaoxiao Zhao, and Li Li. 2020. Active Crowd Counting with Limited Supervision. arXiv preprint arXiv:2007.06334(2020).

Index Terms

  1. Dense or Sparse: Crowd Counting with Binary Supervision

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ICVGIP '22: Proceedings of the Thirteenth Indian Conference on Computer Vision, Graphics and Image Processing
    December 2022
    506 pages
    ISBN:9781450398220
    DOI:10.1145/3571600
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 12 May 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. crowd counting
    2. deep learning
    3. weak supervision

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    • Science and Engineering Research Board (SERB)

    Conference

    ICVGIP'22

    Acceptance Rates

    Overall Acceptance Rate 95 of 286 submissions, 33%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 25
      Total Downloads
    • Downloads (Last 12 months)6
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 14 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media