Abstract
This paper proposes a novel lightweight and fast convolutional neural network to learn a regression model for crowd counting in images captured from drones. The learning system is initially based on a multi-input model trained on two different views of the same input for the task at hand: (i) real-world images; and (ii) corresponding synthetically created “crowd heatmaps”. The synthetic input is intended to help the network focus on the most important parts of the images. The network is trained from scratch on a subset of the VisDrone dataset. During inference, the synthetic path of the network is disregarded resulting in a traditional single-view model optimized for resource-constrained devices. The derived model achieves promising results on the test images, outperforming models developed by state-of-the-art lightweight architectures that can be used for crowd counting and detection.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Boominathan, L., Kruthiventi, S.S., Babu, R.V.: Crowdnet: a deep convolutional network for dense crowd counting. In: Proceedings of the 24th ACM International Conference on Multimedia, pp. 640–644. ACM (2016)
Castellano, G., Castiello, C., Mencar, C., Vessio, G.: Crowd detection for drone safe landing through fully-convolutional neural networks. In: Chatzigeorgiou, A., et al. (eds.) SOFSEM 2020. LNCS, vol. 12011, pp. 301–312. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-38919-2_25
Castellano, G., Castiello, C., Mencar, C., Vessio, G.: Crowd detection in aerial images using spatial graphs and fully-convolutional neural networks. IEEE Access 8, 64534–64544 (2020)
Chaker, R., Al Aghbari, Z., Junejo, I.N.: Social network model for crowd anomaly detection and localization. Pattern Recogn. 61, 266–281 (2017)
Howard, A.G., et al.: Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)
Hsieh, M.R., Lin, Y.L., Hsu, W.H.: Drone-based object counting by spatially regularized regional proposal network. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4145–4153 (2017)
Idrees, H., Saleemi, I., Seibert, C., Shah, M.: Multi-source multi-scale counting in extremely dense crowd images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2547–2554 (2013)
Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Kok, V.J., Lim, M.K., Chan, C.S.: Crowd behavior analysis: a review where physics meets biology. Neurocomputing 177, 342–362 (2016)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Küchhold, M., Simon, M., Eiselein, V., Sikora, T.: Scale-adaptive real-time crowd detection and counting for drone images. In: 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 943–947. IEEE (2018)
Liang, R., Zhu, Y., Wang, H.: Counting crowd flow based on feature points. Neurocomputing 133, 377–384 (2014)
Lin, T.Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Liu, W., Lis, K.M., Salzmann, M., Fua, P.: Geometric and physical constraints for drone-based head plane crowd density estimation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). No. CONF, IEEE/RSJ (2019)
Lloyd, K., Rosin, P.L., Marshall, D., Moore, S.C.: Detecting violent and abnormal crowd activity using temporal analysis of grey level co-occurrence matrix (GLCM)-based texture measures. Mach. Vis. Appl. 28, 361–371 (2017). https://doi.org/10.1007/s00138-017-0830-x
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)
Redmon, J., Farhadi, A.: Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767 (2018)
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv 2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 618–626 (2017)
Sindagi, V.A., Patel, V.M.: CNN-based cascaded multi-task learning of high-level prior and density estimation for crowd counting. In: 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–6. IEEE (2017)
Sindagi, V.A., Patel, V.M.: A survey of recent advances in CNN-based single image crowd counting and density estimation. Pattern Recogn. Lett. 107, 3–16 (2018)
Tzelepi, M., Tefas, A.: Human crowd detection for drone flight safety using convolutional neural networks. In: 2017 25th European Signal Processing Conference (EUSIPCO), pp. 743–747. IEEE (2017)
Tzelepi, M., Tefas, A.: Graph embedded convolutional neural networks in human crowd detection for drone flight safety. IEEE Trans. Emerg. Topics Comput. Intell. (2019)
Valavanis, K.P., Vachtsevanos, G.J.: Handbook of Unmanned Aerial Vehicles. Springer, Heidelberg (2015). https://doi.org/10.1007/978-90-481-9707-1
Xing, J., Ai, H., Liu, L., Lao, S.: Robust crowd counting using detection flow. In: 2011 18th IEEE International Conference on Image Processing, pp. 2061–2064. IEEE (2011)
Yu, J., Wang, M., Tao, D.: Semisupervised multiview distance metric learning for cartoon synthesis. IEEE Trans. Image Process. 21(11), 4636–4648 (2012)
Zhang, C., Li, H., Wang, X., Yang, X.: Cross-scene crowd counting via deep convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 833–841 (2015)
Zhao, J., Xijiong, X., Xu, X., Sun, S.: Multi-view learning overview: recent progress and new challenges. Inf. Fus. 38, 43–54 (2017). https://doi.org/10.1016/j.inffus.2017.02.007
Zhu, P., et al.: VisDrone-VID2019: the vision meets drone object detection in video challenge results. In: Proceedings of the IEEE International Conference on Computer Vision Workshops (2019)
Zhu, P., Wen, L., Bian, X., Ling, H., Hu, Q.: Vision meets drones: A challenge. arXiv preprint arXiv:1804.07437 (2018)
Acknowledgement
This work was supported by the Italian Ministry of Education, University and Research within the “RPASInAir” Project under Grant PON ARS01_00820.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Castellano, G., Castiello, C., Cianciotta, M., Mencar, C., Vessio, G. (2020). Multi-view Convolutional Network for Crowd Counting in Drone-Captured Images. In: Bartoli, A., Fusiello, A. (eds) Computer Vision – ECCV 2020 Workshops. ECCV 2020. Lecture Notes in Computer Science(), vol 12538. Springer, Cham. https://doi.org/10.1007/978-3-030-66823-5_35
Download citation
DOI: https://doi.org/10.1007/978-3-030-66823-5_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-66822-8
Online ISBN: 978-3-030-66823-5
eBook Packages: Computer ScienceComputer Science (R0)