Skip to main content
Springer Nature Link
Log in

Generating Music for Video Games with Real-Time Adaptation to Gameplay Pace

  • Conference paper
  • First Online:
Intelligent Information and Database Systems (ACIIDS 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13995))

Included in the following conference series:

  • 558 Accesses

Abstract

The paper aimed to develop an automatic music generation method for video games that could create various types of music to enhance player immersion and experience, while also being a more cost-effective alternative to human-composed music. The issue of automatic music generation is an interdisciplinary research area that combines topics such as artificial intelligence, music theory, art history, sound engineering, signal processing, and psychology. As a result, the literature to review is vast. Musical compositions can be analyzed from various perspectives, and their applications are extensive, including films, games, and advertisements. Specific methods of music generation may perform better only in a narrow field and range, although we rarely encounter fully computer-generated music nowadays. The proposed algorithm, which utilizes RNN and 4 parameters to control the generation process, was implemented using PyTorch and real-time communication with the game was established using the WebSocket protocol. The algorithm was tested with 14 players who played four levels, each with different background music that was either composed or live-generated. The results showed that the generated music was enjoyed more by the players than the composed music. After implementing improvements from the first round of play tests, all of the generated music received better evaluations than the composed looped music in the second run.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.vgmusic.com/.

  2. 2.

    https://github.com/lucasnfe/adl-piano-midi.

  3. 3.

    http://extras.humdrum.org/man/keycor/.

References

  1. Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5(2), 157–166 (1994)

    Article  Google Scholar 

  2. Briot, J.P., Hadjeres, G., Pachet, F.D.: Deep learning techniques for music generation-a survey. arXiv preprint arXiv:1709.01620 (2017)

  3. Brown, D.: Mezzo: an adaptive, real-time composition program for game soundtracks. In: Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, vol. 8, pp. 68–72 (2012)

    Google Scholar 

  4. Cho, K., Van Merriënboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259 (2014)

  5. Colombo, F., Seeholzer, A., Gerstner, W.: Deep artificial composer: a creative neural network model for automated melody generation. In: Correia, J., Ciesielski, V., Liapis, A. (eds.) EvoMUSART 2017. LNCS, vol. 10198, pp. 81–96. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-55750-2_6

    Chapter  Google Scholar 

  6. Ferreira, L., Lelis, L., Whitehead, J.: Computer-generated music for tabletop role-playing games. In: Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, vol. 16, pp. 59–65 (2020)

    Google Scholar 

  7. Gunawan, A., Iman, A., Suhartono, D.: Automatic music generator using recurrent neural network. Int. J. Comput. Intell. Syst. 13, 645 (2020). https://doi.org/10.2991/ijcis.d.200519.001

    Article  Google Scholar 

  8. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  9. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  10. Krumhansl, C.L.: Cognitive Foundations of Musical Pitch. Oxford University Press, Oxford (2001)

    Book  Google Scholar 

  11. Meade, N., Barreyre, N., Lowe, S.C., Oore, S.: Exploring conditioning for generative music systems with human-interpretable controls. arXiv preprint arXiv:1907.04352 (2019)

  12. Oore, S., Simon, I., Dieleman, S., Eck, D., Simonyan, K.: This time with feeling: learning expressive musical performance. Neural Comput. Appl. 32(4), 955–967 (2020)

    Article  Google Scholar 

  13. Russell, J.A.: A circumplex model of affect. J. Pers. Soc. Psychol. 39(6), 1161 (1980)

    Article  Google Scholar 

  14. Scirea, M., Eklund, P., Togelius, J., Risi, S.: Evolving in-game mood-expressive music with metacompose. In: Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion, pp. 1–8 (2018)

    Google Scholar 

  15. Zheng, K., et al.: Emotionbox: a music-element-driven emotional music generation system using recurrent neural network. arXiv preprint arXiv:2112.08561 (2021)

Download references

Author information

Authors and Affiliations

  1. Faculty of Information and Communication Technology, Wroclaw University of Science and Technology, wybrzeże Wyspiańskiego 27, 50-370, Wroclaw, Poland

    Marek Kopel, Dawid Antczak & Maciej Walczyński

Authors
  1. Marek Kopel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dawid Antczak
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maciej Walczyński
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marek Kopel .

Editor information

Editors and Affiliations

  1. Wrocław University of Science and Technology, Wrocław, Poland

    Ngoc Thanh Nguyen

  2. King Mongkut’s Institute of Technology Ladkrabang, Bangkok, Thailand

    Siridech Boonsang

  3. Iwate Prefectural University Iwate, Iwate, Japan

    Hamido Fujita

  4. Wroclaw University of Science and Technology, Wrocław, Poland

    Bogumiła Hnatkowska

  5. National University of Kaohsiung, Kaohsiung, Taiwan

    Tzung-Pei Hong

  6. King Mongkut’s Institute of Technology Ladkrabang, Bangkok, Thailand

    Kitsuchart Pasupa

  7. Malaysia Japan International Institute of Technology, Kuala Lumpur, Malaysia

    Ali Selamat

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kopel, M., Antczak, D., Walczyński, M. (2023). Generating Music for Video Games with Real-Time Adaptation to Gameplay Pace. In: Nguyen, N.T., et al. Intelligent Information and Database Systems. ACIIDS 2023. Lecture Notes in Computer Science(), vol 13995. Springer, Singapore. https://doi.org/10.1007/978-981-99-5834-4_21

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-5834-4_21

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-5833-7

  • Online ISBN: 978-981-99-5834-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation