Abstract
With recent developments in artificial intelligence, it is possible to generate human motion using deep learning. In this paper, a transformer deep learning algorithm is investigated to generate improvisation dance motions for the Another Kind of Blue (AKOB) data set. AKOB is an innovative dance group, located in The Hague, Netherlands, with a specialization in combining modern dance and technology. For this study, AKOB recorded various dance movements with different pieces of music using a motion capture system. This data is used to train a transformer network and generate sequences of improvisational dance using seed motions and musical input. The produced movements are visualized and compared to the ground truth of human motions to examine their quality. The results show possible human positions, but the speed of motions is a lot compared to the music. Also, sometimes the transition from one position to another is not feasible.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kale, G.V., Patil, V.H.: A study of vision based human motion recognition and analysis. Int. J. Ambient Comp. Intellig. 7(2), 75–92 (2016)
Mallick, T., Das, P.P., Majumdar, A.K.: Posture and sequence recognition for Bharatanatyam dance performances using machine learning approach. arXiv (2019)
Ahn, H., Kim, J., Kim, K., Oh, S.: Generative autoregressive networks for 3D dancing move synthesis from music. IEEE Robutics and Automation Letters 5(2), 3501–3508 (2020)
Another Kind of blue (2022). [Online]. Available: https://www.anotherkindofblue.nl/en/
Holden, D., Saito, J., Komura, T.: A deep learning framework for character motion synthesis and editing. ACM Transactions on Graphics (TOG) 35(4), 1–11 (2016)
Hernandez, A., Gall, J., Moreno-Noguer, F.: Human motion prediction via spatio-temporal in-painting. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 7134–7143 (2019)
Taylor, G.W., Hinton, G.E.: Factored conditional restricted boltzmann machines for modeling motion style. In: Proceedings of the 26th annual international conference on machine learning, pp. 1025–1032 (2009)
Du, X., Vasudevan, R., Matthew, R.J.: Bio-lstm: A biomechanically inspired recurrent neural network for 3-d pedestrian pose and gait prediction. IEEE Robotics and Automation Letters 4(2), 1501–1508 (2019)
Aksan, E., Cao, P., Kaufmann, M., Hilliges, O.: A spatio-temporal transformer for 3d human motion prediction. arXiv preprint arXiv:2004.08692 (2020)
Leijnen, S., Veen, F.V.: The Neural Network Zoo. In: Proceedings, vol. 47(1), no. 9 (2020)
Bengio, S., Vinyals, O., Jaitly, N., Shazeer, N.: Scheduled sampling for sequence prediction with recurrent neural networks. In: Advances in neural information processing systems (2015)
Theater de storm (2022). [Online]. Available: https://www.theaterdestorm.nl/voorstelling/another-kind-of-blue/
Alemi, O., Franc¸oise, J., Pasquier, P.: Real-time music-driven dance movement generation using artificial neural networks. Networks 8(17), 26 (2017)
Ginosar, S., Bar, A., Kohavi, G., Chan, C., Owens, A., Malik, J.: Learning individual styles of conversational gesture. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3497–3506 (2019)
Huang, Y., Kaufmann, M., Aksan, E., Black, M.J., Hilliges, O., Pons-Moll, G.: Deep inertial poser learning to reconstruct human pose from sparseinertial measurements in real time. ACM Transactions on Graphics 37(6), 1–15 (2018). 185
AIST Dance Video Database (May 2022). [Online]. Available: https://aistdancedb.ongaaccel.jp/
Li, R., Yang, S., Ross, D.A., Kanazawa, A.: AI Choreographer: Music conditioned 3D dance generation with AIST++. In: IEEE/CVF International Conference on Computer Vision (ICCV), pp. 13381–13392 (2021)
Optitrack Wiki (May 2022). [Online]. Available: https://v30.wiki.optitrack.com/index.php?title=Quick_Start_Guide:_Getting_Started
Loper, M., Mahmood, N., Romero, J., Moll, G.P., Black, M.J.: SMPL: a skinned multi-person linear model. ACM transactions on graphics (TOG) 34(6), 1–16 (2015)
Codetd: Sorting of joint points in common human hinge joint point data sets (2020). [Online]. Available: https://www.codetd.com/en/article/11819052. Accessed July 2022
Librosa (2022). [Online]. Available: https://librosa.org/
McFee, B., et al.: librosa, Audio and music signal analysis in python. In: Proceedings of the 14th python in science conference, pp. 18–25 (2015)
Vaswani, A., et al.: Attention is all you need. Advances in neural information processing systems 30 (2017)
Sun, C., Baradel, F., Murphy, K., Schmid, C.: Learning video representations using contrastive bidirectional transformer. arXiv preprint arXiv:1906.05743 (2019)
Acknowledgements
We would like to express our gratitude to David Middendorp and the Another Kind of Blue crew for their participation in this project. We would also like to thank Bas van der Linden for his technical advice.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Asadi, Z., Moons, J., Leijnen, S. (2023). Next Level Choreography: Applying a Transformer Network to Generate Improvised Dance Motions. In: Brooks, A.L. (eds) ArtsIT, Interactivity and Game Creation. ArtsIT 2022. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 479. Springer, Cham. https://doi.org/10.1007/978-3-031-28993-4_36
Download citation
DOI: https://doi.org/10.1007/978-3-031-28993-4_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-28992-7
Online ISBN: 978-3-031-28993-4
eBook Packages: Computer ScienceComputer Science (R0)