Abstract
The paper presents a number of recently developed methods for automatic categorization of socio-cognitive crowd behavior from video surveillance data. Sadly and unexpectedly, the recent pandemic outbreak of Covid-19 created a new important niche for such tasks, which otherwise have been rather associated with police monitoring public events or oppressive regimes tightly controlling selected communities. First, we argue that a recently proposed (see [31]) general socio-cognitive categorization of crowd behavior well corresponds to the needs of social distancing monitoring. It is explained how each of four proposed categories represents a different level of social (ir)responsibility in public spaces. Then, several techniques are presented which can be used to perform in real time such a categorization, based only the raw-data inputs (i.e. video-sequences from surveillance cameras). In particular, we discuss: (a) selected detection and tracking aspects for individual people and their groups, (b) practicality of data association combining results of detection and tracking, and (c) mid-level features proposed for neural-network-based classifiers of the behavior categories. Some illustrative results obtained in the developed feasibility studies are also included.
Supported by Khalifa University.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Alyammahi, S., Bhaskar, H., Ruta, D., Al-Mualla, M.: People detection and articulated pose estimation framework for crowded scenes. Knowl.-Based Syst. 131, 83–104 (2017). https://doi.org/10.1016/j.knosys.2017.06.001
Baig, M.W., Barakova, E.I., Marcenaro, L., Regazzoni, C.S., Rauterberg, M.: Bio-inspired probabilistic model for crowd emotion detection. In: 2014 International Joint Conference on Neural Networks (IJCNN), pp. 3966–3973 (2014). https://doi.org/10.1109/IJCNN.2014.6889964
Chakravartula, R., Aparna, V., Chithira, S., Vidhya, B.: A comparative study of vision based human detection techniques in people counting applications. Procedia Comput. Sci. 58, 461–469 (2015)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: 2005 IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 886–893. IEEE (2005). https://doi.org/10.1109/CVPR.2005.177
Dollár, P., Appel, R., Belongie, S., Perona, P.: Fast feature pyramids for object detection. IEEE Trans. Pattern Anal. Mach. Intell. 36(8), 1532–1545 (2014)
Edman, V., Andersson, M., Granström, K., Gustafsson, F.: Pedestrian group tracking using the GM-PHD filter. In: 21st European Signal Processing Conference (EUSIPCO 2013), pp. 1–5. IEEE (2013)
Grant, J.M., Flynn, P.J.: Crowd scene understanding from video: a survey. ACM Trans. Multimed. Comput. Commun. Appl. (TOMM) 13(2), 19 (2017). https://doi.org/10.1145/3052930
Li, T., Chang, H., Wang, M., Ni, B., Hong, R., Yan, S.: Crowded scene analysis: a survey. IEEE Trans. Circuits Syst. Video Technol. 25(3), 367–386 (2015). https://doi.org/10.1109/TCSVT.2014.2358029
Manfredi, M., Vezzani, R., Calderara, S., Cucchiara, R.: Detection of static groups and crowds gathered in open spaces by texture classification. Pattern Recogn. Lett. 44, 39–48 (2014). https://doi.org/10.1016/j.patrec.2013.11.001
Mazzon, R., Poiesi, F., Cavallaro, A.: Detection and tracking of groups in crowd. In: 2013 10th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 202–207, August 2013. https://doi.org/10.1109/AVSS.2013.6636640
Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using social force model. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 935–942, June 2009
Raj, K.S., Poovendran, R.: Pedestrian detection and tracking through hierarchical clustering. In: International Conference on Information Communication and Embedded Systems (ICICES2014), pp. 1–4. IEEE (2014). https://doi.org/10.1109/ICICES.2014.7033991
Redmon, J., Farhadi, A.: Yolo9000: better, faster, stronger. In: 2017 IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7263–7271. IEEE (2017). https://doi.org/10.1109/CVPR.2017.690
Rezatofighi, S.H., Milan, A., Zhang, Z., Shi, Q., Dick, A., Reid, I.: Joint probabilistic data association revisited. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 3047–3055. IEEE (2015). https://doi.org/10.1109/ICCV.2015.349
Shao, J., Change Loy, C., Wang, X.: Scene-independent group profiling in crowd. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2219–2226 (2014). https://doi.org/10.1109/CVPR.2014.285
Shao, J., Kang, K., Change Loy, C., Wang, X.: Deeply learned attributes for crowded scene understanding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4657–4666 (2015)
Solmaz, B., Moore, B.E., Shah, M.: Identifying behaviors in crowd scenes using stability analysis for dynamical systems. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2064–2070 (2012). https://doi.org/10.1109/TPAMI.2012.123
Tomè, D., Monti, F., Baroffio, L., Bondi, L., Tagliasacchi, M., Tubaro, S.: Deep convolutional neural networks for pedestrian detection. Sig. Process. Image Commun. 47, 482–489 (2016)
Wang, X., Li, T., Sun, S., Corchado, J.M.: A survey of recent advances in particle filters and remaining challenges for multitarget tracking. Sensors 17(12) (2017). https://doi.org/10.3390/s17122707
Wen, L., Lei, Z., Lyu, S., Li, S.Z., Yang, M.H.: Exploiting hierarchical dense structures on hypergraphs for multi-object tracking. IEEE Trans. Pattern Anal. Mach. Intell. 38(10), 1983–1996 (2016)
Yang, B., Nevatia, R.: Multi-target tracking by online learning a CRF model of appearance and motion patterns. Int. J. Comput. Vis. 107(2), 203–217 (2014). https://doi.org/10.1007/s11263-013-0666-4
Yang, Y., Ramanan, D.: Articulated human detection with flexible mixtures of parts. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 35(12), 2878–2890 (2013)
Zhan, B., Monekosso, D.N., Remagnino, P., Velastin, S.A., Xu, L.Q.: Crowd analysis: a survey. Mach. Vis. Appl. 19, 345–357 (2008)
Zhang, S., Wang, J., Wang, Z., Gong, Y., Liu, Y.: Multi-target tracking by learning local-to-global trajectory models. Pattern Recogn. 48(2), 580–590 (2015)
Zhang, Y., Qin, L., Ji, R., Yao, H., Huang, Q.: Social attribute-aware force model: exploiting richness of interaction for abnormal crowd detection. IEEE Trans. Circuits Syst. Video Technol. 25(7), 1231–1245 (2015). https://doi.org/10.1109/TCSVT.2014.2355711
Zhang, Y., Qin, L., Yao, H., Huang, Q.: Abnormal crowd behavior detection based on social attribute-aware force model. In: 2012 19th IEEE International Conference on Image Processing, pp. 2689–2692, September 2012
Zhou, B., Wang, X., Tang, X.: Understanding collective crowd behaviors: learning a mixture model of dynamic pedestrian-agents. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2871–2878 (2012)
Zhou, T., Yang, J., Loza, A., Bhaskar, H., Al-Mualla, M.: A crowd modelling framework using fast head detection and shape-aware matching. J. Electron. Imaging 24 (2015)
Zhu, F., Wang, X., Yu, N.: Crowd tracking by group structure evolution. IEEE Trans. Circuits Syst. Video Technol. 28(3), 772–786 (2018). https://doi.org/10.1109/TCSVT.2016.2615460
Zitouni, M.S., Sluzek, A., Bhaskar, H.: CNN-based analysis of crowd structure using automatically annotated training data. In: 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–8, September 2019. https://doi.org/10.1109/AVSS.2019.8909846
Zitouni, M.S., Sluzek, A., Bhaskar, H.: Visual analysis of socio-cognitive crowd behaviors for surveillance: a survey and categorization of trends and methods. Eng. Appl. Artif. Intell. 82, 294–312 (2019). https://doi.org/10.1016/j.engappai.2019.04.012
Zitouni, M.S., Sluzek, A., Bhaskar, H.: Towards understanding socio-cognitive behaviors of crowds from visual surveillance data. Multimed. Tools Appl. 79, 1781–1799 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zitouni, M.S., Śluzek, A. (2020). Video-Surveillance Tools for Monitoring Social Responsibility Under Covid-19 Restrictions. In: Chmielewski, L.J., Kozera, R., Orłowski, A. (eds) Computer Vision and Graphics. ICCVG 2020. Lecture Notes in Computer Science(), vol 12334. Springer, Cham. https://doi.org/10.1007/978-3-030-59006-2_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-59006-2_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59005-5
Online ISBN: 978-3-030-59006-2
eBook Packages: Computer ScienceComputer Science (R0)