Abstract
In the field of crowd behavior analysis, existing methods mainly focus on using local representations inspired by models found in other disciplines (e.g., fluid dynamics and social dynamics) to describe motion patterns. However, less attention is paid to exploiting motion structures (e.g., visual information contained in trajectories) for behavior analysis. In this paper, we consider both local characteristics and global structures of a motion vector field, and propose the Curl and Divergence of motion Trajectories (CDT) descriptors to describe collective motion patterns. To this end, a trajectory-based motion coding algorithm is designed to extract the CDT descriptors. For each motion vector field we construct its conjugate field, in which each vector is perpendicular to the counterpart in the original vector field. The trajectories in the motion and corresponding conjugate fields indicate the tangential and radial motion structures, respectively. By integrating curl (and divergence, respectively) along the tangential paths (and the radial paths, respectively), the CDT descriptors are extracted. We show that the proposed motion descriptors are scale- and rotation-invariant for effective crowd behavior analysis. For concreteness, we apply the CDT descriptors to identify five typical crowd behaviors (lane, clockwise arch, counterclockwise arch, bottleneck and fountainhead) with a pipeline including motion decomposition. Extensive experimental results on two benchmark datasets demonstrate the effectiveness of the CDT descriptors for describing and classifying crowd behaviors.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The source code can be found at the website: http://github.com/shuangseu/CDT_crowd_descriptor.
The detailed parameter setting can be found at the website: http://github.com/shuangseu/CDT_crowd_descriptor.
References
Ali, S. (2013). Measuring flow complexity in videos. In IEEE International Conference on Computer Vision
Ali, S., & Shah, M. (2007). A lagrangian particle dynamics approach for crowd flow segmentation and stability analysis. In IEEE Conference on Computer vision and Pattern Recognition
Ali, S., & Shah, M. (2010). Human action recognition in videos using kinematic features and multiple instance learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(2), 288–303.
Brostow, G. J., & Cipolla, R. (2006). Unsupervised bayesian detection of independent motion in crowds. In IEEE Conference on Computer Vision and Pattern Recognition
Cezar, J., Jacques, S., Musse, S. R., Silveira Jacques Junior, J., & Jung, C. (2010). Crowd analysis using computer vision techniques. IEEE Signal Processing Magazine, 27(5), 66–77.
Chang, C., & Lin, C. (2011). LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(3), 1–27.
Cheriyadat, A., & Radke, R. J. (2008). Detecting dominant motions in dense crowds. IEEE Journal of Selected Topics in Signal Processing, 2(4), 568–581.
Comaniciu, D., & Meer, P. (2002). Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(5), 603–619.
Cong, Y., Yuan, J., & Liu, J. (2013). Abnormal event detection in crowded scenes using sparse representation. Pattern Recognition, 46(7), 1851–1864.
Fukunaga, K., & Hostetler, L. D. (1975). The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Transactions on Information Theory, 21(1), 32–40.
Hu, M., Ali, S., & Shah, M. (2008a). Detecting global motion patterns in complex videos. In IEEE International Conference on Pattern Recognition
Hu, M., Ali, S., & Shah, M. (2008b). Learning motion patterns in crowded scenes using motion flow field. In IEEE International Conference on Pattern Recognition
Kratz, L., & Nishino, K. (2012). Tracking pedestrians using local spatio-temporal motion patterns in extremely crowded scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(5), 987–1002.
Li, R., & Chellappa, R. (2010). Group motion segmentation using a spatio-temporal driving force model. In IEEE Conference on Computer Vision and Pattern Recognition
Li, T., Chang, H., Wang, M., Ni, B., Hong, R., & Yan, S. (2015). Crowded scene analysis: A survey. IEEE Transactions on Circuits and Systems for Video Technology, 25(3), 367–386.
Lin, D., Grimson, E. L., & Fisher, J. W. (2009). Learning visual flows: A Lie algebraic approach. In IEEE Conference on Computer Vision and Pattern Recognition
Lucas, B. D., & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. In International Joint Conference on Artificial Intelligence
Marsden, J. E., & Tromba, A. (2003). Vector calculus. San Francisco: W.H Freeman.
Mehran, R., Oyama, A., & Shah, M. (2009). Abnormal crowd behavior detection using social force model. In IEEE Conference on Computer Vision and Pattern Recognition
Mehran, R., Oyama, A., & Shah, M. (2009). Abnormal crowd behavior detection using social force model. In IEEE Conference on Computer Vision and Pattern Recognition
Saleemi, I., Hartung, L., & Shah, M. (2010). Scene understanding by statistical modeling of motion patterns. In IEEE Conference on Computer Vision and Pattern Recognition, pp 2069–2076
Shao, J., Loy, C. C., & Wang, X. (2014). Scene-independent group profiling in crowd. In IEEE Conference on Computer Vision and Pattern Recognition
Solmaz, B., Moore, B. E., & Shah, M. (2012). Identifying behaviors in crowd scenes using stability analysis for dynamical systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(10), 2064–2070.
Su, H., Yang, H., Zheng, S., Fan, Y., & Wei, S. (2013). The large-scale crowd behavior perception based on spatio-temporal viscous fluid field. IEEE Transactions on Information Forensics and Security, 8(10), 1575–1589.
Wang, H., & Schmid, C. (2013). Action recognition with improved trajectories. In IEEE International Conference on Computer Vision
Wang, H., Kläser, A., Schmid, C., & Liu, C. (2011). Action recognition by dense trajectories. In IEEE Conference on Computer Vsion and Pattern Recognition
Wang, L., Qiao, Y., & Tang, X. (2015). Action recognition with trajectory-pooled deep-convolutional descriptors. In IEEE Conference on Computer Vision and Pattern Recognition
Wang, W., Lin, W., Chen, Y., Wu, J., Wang, J., & Sheng, B. (2014) Finding coherent motions and semantic regions in crowd scenes: A diffusion and clustering approach. In European Conference on Computer Vision
Wu, S., & Wong, H. (2012). Crowd motion partitioning in a scattered motion field. IEEE Transactions on Systems, Man, and Cybernetics, 42(5), 1443–1454.
Wu, S., Moore, B. E., & Shah, M. (2010). Chaotic invariants of lagrangian particle trajectories for anomaly detection in crowded scenes. In IEEE Conference on Computer Vision and Pattern Recognition
Yi, S., Wang, X., Lu, C., & Jia, J. (2014). L0 regularized stationary time estimation for crowd group analysis. In IEEE Conference on Computer Vision and Pattern Recognition
Yi, S., Li, H., & Wang, X. (2015) Understanding pedestrian behaviors from stationary crowd groups. In IEEE Conference on Computer Vision and Pattern Recognition
Zhan, B., Monekosso, D. N., Remagnino, P., & Velastin Sa, Xu L Q. (2008). Crowd analysis: A survey. Machine Vision and Applications, 19(5–6), 345–357.
Zhao, X., & Medioni, G. G. (2011) Robust unsupervised motion pattern inference from video and applications. In IEEE International Conference on Computer Vision
Zhou, B., Wang, X., & Tang, X. (2011) Random field topic model for semantic region analysis in crowded scenes from tracklets. In IEEE Conference on Computer Vision and Pattern Recognition
Zhou, B., Tang, X., & Wang, X. (2012a) Coherent filtering: Detecting coherent motions from crowd clutters. In European Conference on Computer Vision
Zhou, B., Wang, X., & Tang, X. (2012b). Understanding collective crowd behaviors: Learning a mixture model of dynamic pedestrian-agents. In IEEE Conference on Computer Vision and Pattern Recognition
Zhou, B., Tang, X., & Wang, X. (2013). Measuring crowd collectiveness. In IEEE Conference on Computer Vision and Pattern Recognition
Acknowledgements
We thank Dr. Berkan Solmaz for sharing the UCF crowd dataset and discussing technical details, as well as Jing Shao and Bolei Zhou for providing the CUHK crowd dataset. This work is supported in part by NSFC 61671289, 61171172, 61102099, 61571261 and 61521062, STCSM Grant 15DZ1207403, and NSF CAREER Grant 1149783.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Jiri Matas.
Rights and permissions
About this article
Cite this article
Wu, S., Yang, H., Zheng, S. et al. Crowd Behavior Analysis via Curl and Divergence of Motion Trajectories. Int J Comput Vis 123, 499–519 (2017). https://doi.org/10.1007/s11263-017-1005-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-017-1005-y