Abstract
The existing methods on video synthesis have succeeded in generating higher quality videos by using guide information such as human pose skeletons, segmentation masks and optical flows as auxiliary information. Some existing video generation methods on human motion adopts a two-step video generation consisting of generation of pose sequences and video generation from pose sequences. In this paper, we focus on the first stage, the generation of pose sequences, in the whole processing of video generation of human motion. We incorporate a Graph Convolutional Network (GCN) and an initial pose generator into the model to model poses more explicitly and to generate pose sequences naturally. The experimental results show that the proposed method can generate better quality pose sequences than the conventional methods by improving the initial pose generation and introducing GCN.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Brock, A., Donahue, J., Simonyan, K.: Large scale GAN training for high fidelity natural image synthesis. In: Proceedings of International Conference on Learning Representations (2019). https://openreview.net/forum?id=B1xsqj09Fm
Cai, H., Bai, C., Tai, Y., Tang, C.: Deep video generation, prediction and completion of human action sequences. In: Proceedings of of European Conference on Computer Vision, pp. 366–382 (2018)
Clark, A., Donahue, J., Simonyan, K.: Adversarial video generation on complex datasets. arXiv preprint arXiv:1907.06571 (2019)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)
Guo, C., et al.: Action2motion: conditioned generation of 3D human motions. In: Proceedings of ACM International Conference Multimedia (2020)
Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: Proc. of IEEE Computer Vision and Pattern Recognition, pp. 4401–4410 (2019)
Kingma, D., Welling, M.: Auto-encoding variational Bayes. In: Proceedings of International Conference on Learning Representations (2014)
Liu, Z., Zhang, H., Chen, Z., Wang, Z., Ouyang, W.: Disentangling and unifying graph convolutions for skeleton-based action recognition. In: Proceedings of IEEE Computer Vision and Pattern Recognition, pp. 143–152 (2020)
Mallya, A., Wang, T.C., Sapra, K., Liu, M.Y.: World-consistent video-to-video synthesis. In: Proceedings of European Conference on Computer Vision (2020)
Niepert, M., Ahmed, M., Kutzkov, K.: Learning convolutional neural networks for graphs. In: Proceedings of International Conference on Machine Learning, pp. 2014–2023 (2016)
Razavi, A., van den Oord, A., Vinyals, O.: Generating diverse high-fidelity images with VQ-VAE-2. In: Advances in Neural Information Processing Systems, pp. 14866–14876 (2019)
Ren, Y., Li, G., Liu, S., Li, T.H.: Deep spatial transformation for pose-guided person image generation and animation. IEEE Trans. Image Process. (2020)
Saito, M., Matsumoto, E., Saito, S.: Temporal generative adversarial nets with singular value clipping. In: Proceedings of IEEE International Conference on Computer Vision, pp. 2830–2839 (2017)
Tulyakov, S., Liu, M., Yang, X., Kautz, J.: MoCoGAN: decomposing motion and content for video generation. In: Proceedings of IEEE Computer Vision and Pattern Recognition (2018)
Vondrick, C., Pirsiavash, H., Torralba, A.: Generating videos with scene dynamics. In: Advances in Neural Information Processing Systems, vol. 29, pp. 613–621 (2016)
Wang, T., et al.: Video-to-video synthesis. In: Advances in Neural Information Processing Systems (2018)
Xu, J., Xu, H., Ni, B., Yang, X., Wang, X., Darrell, T.: Hierarchical style-based networks for motion synthesis. In: Proceedings of European Conference on Computer Vision (2020)
Zhao, L., Peng, X., Tian, Y., Kapadia, M., Metaxas, D.N.: Semantic graph convolutional networks for 3D human pose regression. In: Proceedings of IEEE Computer Vision and Pattern Recognition, pp. 3420–3430 (2019)
Acknowledgement
This work was supported by JSPS KAKENHI Grant Number 17H06100 and 21H05812.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Terauchi, K., Yanai, K. (2022). Pose Sequence Generation with a GCN and an Initial Pose Generator. In: Wallraven, C., Liu, Q., Nagahara, H. (eds) Pattern Recognition. ACPR 2021. Lecture Notes in Computer Science, vol 13188. Springer, Cham. https://doi.org/10.1007/978-3-031-02375-0_31
Download citation
DOI: https://doi.org/10.1007/978-3-031-02375-0_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-02374-3
Online ISBN: 978-3-031-02375-0
eBook Packages: Computer ScienceComputer Science (R0)