Skip to main content
SpringerLink
Log in

Predicting Marimba Stickings Using Long Short-Term Memory Neural Networks

  • Conference paper
  • First Online:
AI 2022: Advances in Artificial Intelligence (AI 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13728))

Included in the following conference series:

  • 1404 Accesses

Abstract

In marimba music, ‘stickings’ are the choices of mallets used to strike each note. Stickings significantly influence both the physical facility and expressive quality of the music performance. Choosing ‘good’ stickings and evaluating one’s stickings are complex choices, often relying vaguely on trial-and-error. Machine learning (ML) approaches, particularly with advances in sequence-to-sequence techniques, have proved suited for similar complex classification problems, motivating their application in our study. We address the sticking problem by developing Long Short-Term Memory (LSTM) models to generate stickings in 4-mallet marimba music trained on exercises from Leigh Howard Stevens’ Method of Movement for Marimba. Model performance was measured under a range of metrics to account for multiple sticking possibilities, with LSTM models achieving a maximum average micro-accuracy of 97.3%. Finally, we discuss qualitative observations in sticking predictions and limitations of this study and provide direction for further development in this field.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Al Kasimi, A., Nichols, E., Raphael, C.: Automatic fingering system (afs). In: Poster presentation at ISMIR, London (2005)

    Google Scholar 

  2. Al Kasimi, A., Nichols, E., Raphael, C.: A simple algorithm for or automatic generation of polyphonic piano fingerings (2007)

    Google Scholar 

  3. Balliauw, M., Herremans, D., Palhazi Cuervo, D., Sörensen, K.: A variable neighborhood search algorithm to generate piano fingerings for polyphonic sheet music. Int. Trans. Oper. Res. 24(3), 509–535 (2017)

    Article  MATH  Google Scholar 

  4. Barbancho, A.M., Klapuri, A., Tardon, L.J., Barbancho, I.: Automatic transcription of guitar chords and fingering from audio. IEEE Trans. Audio Speech Lang. Process. 20(3), 915–921 (2012). https://doi.org/10.1109/TASL.2011.2174227

    Article  Google Scholar 

  5. Bretan, P.M.: Towards an embodied musical mind: generative algorithms for robotic musicians. Ph.D. thesis, Georgia Institute of Technology (2017)

    Google Scholar 

  6. Briot, J.P., Hadjeres, G., Pachet, F.: Deep Learning Techniques for Music Generation. Springer, Heidelberg (2020). https://doi.org/10.1007/978-3-319-70163-9

    Book  Google Scholar 

  7. Briot, J.P., Pachet, F.: Deep learning for music generation: challenges and directions. Neural Comput. Appl. 32(4), 981–993 (2020)

    Article  Google Scholar 

  8. Burlet, G., Fujinaga, I.: Robotaba guitar tablature transcription framework. In: Proceedings of the 14th International Society for Music Information Retrieval Conference, ISMIR, Curitiba, Brazil, pp. 517–522 (2013)

    Google Scholar 

  9. Choi, K., Fazekas, G., Cho, K., Sandler, M.: A tutorial on deep learning for music information retrieval. arXiv preprint arXiv:1709.04396 (2017)

  10. Choi, K., Fazekas, G., Sandler, M., Cho, K.: Convolutional recurrent neural networks for music classification. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2392–2396. IEEE (2017)

    Google Scholar 

  11. Chollet, F., et al.: Keras (2015). https://github.com/fchollet/keras

  12. Coca, A.E., Corrêa, D.C., Zhao, L.: Computer-aided music composition with LSTM neural network and chaotic inspiration. In: The 2013 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE (2013)

    Google Scholar 

  13. Corrêa, D.C., Levada, A.L., Saito, J.H., Mari, J.F.: Neural network based systems for computer-aided musical composition: supervised x unsupervised learning. In: Proceedings of the 2008 ACM Symposium on Applied Computing, pp. 1738–1742 (2008)

    Google Scholar 

  14. Cuthbert, M., Ariza, C., Hogue, B., Oberholtzer, J.W.: music21 (version 5.7.2) [python package] (2006–2021). https://web.mit.edu/music21

  15. Johnson, D., Damian, G.T.D.: Detecting hand posture in piano playing using depth data. Comput. Music J. 43(1), 59–78 (2019). http://muse.jhu.edu/article/746693

  16. De Prisco, R., Zaccagnino, G., Zaccagnino, R.: A differential evolution algorithm assisted by ANFIS for music fingering. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) EC/SIDE -2012. LNCS, vol. 7269, pp. 48–56. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29353-5_6

    Chapter  Google Scholar 

  17. Eck, D., Lapalme, J.: Learning musical structure directly from sequences of music. University of Montreal, Department of Computer Science, CP 6128, 48 (2008)

    Google Scholar 

  18. Eck, D., Schmidhuber, J.: Finding temporal structure in music: blues improvisation with LSTM recurrent networks. In: Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing, pp. 747–756. IEEE (2002)

    Google Scholar 

  19. Eck, D., Schmidhuber, J.: A first look at music composition using LSTM recurrent neural networks. Istituto Dalle Molle Di Studi Sull Intelligenza Artificiale 103, 48 (2002)

    Google Scholar 

  20. Hart, M., Bosch, R., Tsai, E.: Finding optimal piano fingerings. UMAP J. 21(2), 167–177 (2000)

    Google Scholar 

  21. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  22. Humphrey, E.J., Bello, J.P.: From music audio to chord tablature: teaching deep convolutional networks to play guitar. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6974–6978 (2014). https://doi.org/10.1109/ICASSP.2014.6854952

  23. Liu, I., Ramakrishnan, B., et al.: Bach in 2014: music composition with recurrent neural network. arXiv preprint arXiv:1412.3191 (2014)

  24. Lyu, Q., Wu, Z., Zhu, J.: Polyphonic music modelling with LSTM-RTRBM. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp. 991–994 (2015)

    Google Scholar 

  25. Mistler, E.: Generating Guitar Tablatures with Neural Networks. University of Edinburgh, Thesis (2017)

    Google Scholar 

  26. Miura, M., Hirota, I., Hama, N., Yanagida, M.: Constructing a system for finger-position determination and tablature generation for playing melodies on guitars. Syst. Comput. Japan 35(6), 10–19 (2004). https://doi.org/10.1002/scj.10609. https://search.ebscohost.com/login.aspx?direct=true &db=iih &AN=13217635 &site=ehost-live

  27. Musafia, J.: The art of fingering in piano playing. MCA Music (1971)

    Google Scholar 

  28. MuseScore: MuseScore (version 3.1.0.7078) [computer software] (2019). https://musescore.org

  29. Nakamura, E., Ono, N., Sagayama, S.: Merged-output hmm for piano fingering of both hands. In: ISMIR, pp. 531–536 (2014)

    Google Scholar 

  30. Nakamura, E., Saito, Y., Yoshii, K.: Statistical learning and estimation of piano fingering. Inf. Sci. 517, 68–85 (2020). https://doi.org/10.1016/j.ins.2019.12.068. http://www.sciencedirect.com/science/article/pii/S0020025519311879

  31. Parncutt, R., Sloboda, J.A., Clarke, E.F., Raekallio, M., Desain, P.: An ergonomic model of keyboard fingering for melodic fragments. Music Percept. 14(4), 341–382 (1997)

    Article  Google Scholar 

  32. Python: Python (version 3.7.3) [computer software] (2001–2021). https://python.org

  33. Ramos, J.V., Ramos, A.S., Silla, C.N., Sanches, D.S.: An evaluation of different evolutionary approaches applied in the process of automatic transcription of music scores into tablatures. In: 2016 IEEE 28th International Conference on Tools with Artificial Intelligence (ICTAI), pp. 663–669 (2016). https://doi.org/10.1109/ICTAI.2016.0106

  34. Savery, R., Weinberg, G.: Shimon the robot film composer and deepscore. In: Proceedings of Computer Simulation of Musical Creativity, p. 5 (2018)

    Google Scholar 

  35. Sayegh, S.I.: Fingering for string instruments with the optimum path paradigm. Comput. Music Jou. 13(3), 76–84 (1989). https://doi.org/10.2307/3680014. http://www.jstor.org.ezproxy.library.uwa.edu.au/stable/3680014

  36. Schedl, M.: Deep learning in music recommendation systems. Front. Appl. Math. Stat. 5, 44 (2019)

    Article  Google Scholar 

  37. Sigtia, S., Benetos, E., Dixon, S.: An end-to-end neural network for polyphonic piano music transcription. IEEE/ACM Trans. Audio Speech Lang. Process. 24(5), 927–939 (2016)

    Article  Google Scholar 

  38. Stevens, L.H.: Method of Movement for Marimba: With 590 Exercises. Marimba Productions, Inc., Neptune City (2005)

    Google Scholar 

  39. Sturm, B.L., Santos, J.F., Ben-Tal, O., Korshunova, I.: Music transcription modelling and composition using deep learning. arXiv preprint arXiv:1604.08723 (2016)

  40. Takegawa, Y., Terada, T., Nishio, S.: Design and implementation of a real-time fingering detection system for piano performance. In: ICMC (2006)

    Google Scholar 

  41. Tuohy, D.R., Potter, W.: Guitar tablature creation with neural networks and distributed genetic search. In: Proceedings of the 19th International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert Systems, IEA-AIE06, Annecy, France (2006)

    Google Scholar 

  42. Van Den Oord, A., Dieleman, S., Schrauwen, B.: Deep content-based music recommendation. In: Neural Information Processing Systems Conference (NIPS 2013), vol. 26. Neural Information Processing Systems Foundation (NIPS) (2013)

    Google Scholar 

  43. Walter, D.W.: The Performance of Contrapuntal Music on the Marimba and Vibraphone. Ph.D. thesis, Temple University (1984)

    Google Scholar 

  44. Wang, X., Wang, Y.: Improving content-based and hybrid music recommendation using deep learning. In: Proceedings of the 22nd ACM international conference on Multimedia, pp. 627–636 (2014)

    Google Scholar 

  45. Yang, N., Savery, R., Sankaranarayanan, R., Zahray, L., Weinberg, G.: Mechatronics-driven musical expressivity for robotic percussionists. arXiv preprint arXiv:2007.14850 (2020)

  46. Yazawa, K., Itoyama, K., Okuno, H.G.: Automatic transcription of guitar tablature from audio signals in accordance with player’s proficiency. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3122–3126 (2014). https://doi.org/10.1109/ICASSP.2014.6854175

  47. Ycart, A., Benetos, E., et al.: A study on LSTM networks for polyphonic music sequence modelling. In: ISMIR (2017)

    Google Scholar 

  48. Yonebayashi, Y., Kameoka, H., Sagayama, S.: Automatic decision of piano fingering based on a hidden markov models. In: IJCAI, vol. 7, pp. 2915–2921 (2007)

    Google Scholar 

  49. Yu, Y., Luo, S., Liu, S., Qiao, H., Liu, Y., Feng, L.: Deep attention based music genre classification. Neurocomputing 372, 84–91 (2020)

    Article  Google Scholar 

  50. Zeltsman, N.: Four-Mallet Marimba Playing: A Musical Approach for All Levels. H. Leonard, Milwaukee, WI (2003)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Conservatorium of Music, The University of Western Australia, Crawley, WA, 6009, Australia

    Jet Kye Chong

  2. Department of Computer Science and Software Engineering, The University of Western Australia, Crawley, WA, 6009, Australia

    Débora Corrêa

  3. ARC Centre for Transforming Maintenance Through Data Science, The University of Western Australia, Crawley, WA, 6009, Australia

    Débora Corrêa

Authors
  1. Jet Kye Chong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Débora Corrêa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jet Kye Chong .

Editor information

Editors and Affiliations

  1. University of New South Wales, Sydney, NSW, Australia

    Haris Aziz

  2. University of Western Australia, Perth, WA, Australia

    Débora Corrêa

  3. University of Western Australia, Perth, WA, Australia

    Tim French

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chong, J.K., Corrêa, D. (2022). Predicting Marimba Stickings Using Long Short-Term Memory Neural Networks. In: Aziz, H., Corrêa, D., French, T. (eds) AI 2022: Advances in Artificial Intelligence. AI 2022. Lecture Notes in Computer Science(), vol 13728. Springer, Cham. https://doi.org/10.1007/978-3-031-22695-3_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-22695-3_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-22694-6

  • Online ISBN: 978-3-031-22695-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation