Abstract
ECO [20] is a newly proposed efficient video classification network that combines 2D and 3D networks. On this basis, we try to extract the global features composed of inter-frame relationships and fuse the global features with the initial frame feature, which can enhance the learning of video features to achieve better classification results. In this way, the new frame feature not only contains the information of itself but also contains the global feature which is related to other frames., so the module can more easily learn about distinguishing features. The main contribution of this paper is proposing a module for extracting global features to enhance video feature learning. It is very lightweight and basically does not add computation to the model. At the same time, it is flexible and easy to generalize to various network structures. Finally, based on the baseline the classification accuracy is increased by 0.8% on the HMDB51 dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Carreira, J., Zisserman, A.: Quo vadis, action recognition? A new model and the kinetics dataset. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4724–4733. IEEE (2017)
Chen, Y., Kalantidis, Y., Li, J., Yan, S., Feng, J.: Multi-fiber networks for video recognition. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 352–367 (2018)
Hu, H., Gu, J., Zhang, Z., Dai, J., Wei, Y.: Relation networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3588–3597 (2018)
Hara, K., Kataoka, H., Satoh, Y.: Can spatiotemporal 3D CNNs retrace the history of 2D CNNs and ImageNet? In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6546–6555 (2018)
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., Serre, T.: HMDB: a large video database for human motion recognition. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 2556–2563. IEEE (2011)
Kim, J., Ricci, M., Serre, T.: Not-So-CLEVR: visual relations strain feedforward neural networks (2018)
Lin, J., Gan, C., Han, S.: Temporal shift module for efficient video understanding. arXiv preprint arXiv:1811.08383 (2018)
Qiu, Z., Yao, T., Mei, T.: Learning spatio-temporal representation with pseudo-3D residual networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 5534–5542. IEEE, October 2017
Santoro, A., Raposo, D., Barrett, D.G., Malinowski, M., Pascanu, R., Battaglia, P., Lillicrap, T.: A simple neural network module for relational reasoning. In: Advances in Neural Information Processing Systems, pp. 4967–4976 (2017)
Su, B., Zhou, J., Ding, X., Wang, H., Wu, Y.: Hierarchical dynamic parsing and encoding for action recognition. In: European Conference on Computer Vision (ECCV), pp. 202–217 (2016)
Su, B., Zhou, J., Ding, X., Wu, Y.: Unsupervised hierarchical dynamic parsing and encoding for action recognition. IEEE Trans. Image Process. 26, 5784–5799 (2017)
Su, B., Hua, G.: Order-preserving Wasserstein distance for sequence matching. In: Computer Vision and Pattern Recognition (CVPR), pp. 1049–1057 (2017)
Su, B., Hua, G.: Order-preserving optimal transport for distances between sequences. IEEE Trans. Pattern Anal. Mach. Intell. (2018)
Tran, D., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.: Learning spatiotemporal features with 3D convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4489–4497 (2015)
Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., Paluri, M.: A closer look at spatiotemporal convolutions for action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6450–6459 (2018)
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., Van Gool, L.: Temporal segment networks: towards good practices for deep action recognition. In: European Conference on Computer Vision, pp. 20–36. Springer, Cham (2016)
Xie, S., Sun, C., Huang, J., Tu, Z., Murphy, K.: Rethinking spatiotemporal feature learning: speed-accuracy trade-offs in video classification. In: European Conference on Computer Vision, pp. 318–335. Springer, Cham (2018)
Zhou, B., Andonian, A., Oliva, A., Torralba, A.: Temporal relational reasoning in videos. In: European Conference on Computer Vision, pp. 831–846. Springer, Cham (2018)
Zolfaghari, M., Singh, K., Brox, T.: ECO: efficient convolutional network for online video understanding. arXiv preprint arXiv:1804.09066 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Kong, M., Lv, P. (2020). Global Features of Fused Frame Relationships Help Video Classification. In: Liu, Y., Wang, L., Zhao, L., Yu, Z. (eds) Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery. ICNC-FSKD 2019. Advances in Intelligent Systems and Computing, vol 1074. Springer, Cham. https://doi.org/10.1007/978-3-030-32456-8_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-32456-8_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32455-1
Online ISBN: 978-3-030-32456-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)