Abstract
This report discusses the behavioural learning properties of a musical agent learning to generate a two-part counterpoint using SARSA, one of the on-policy temporal difference learning approaches. The policy was learned using hand-crafted rules describing the desired characteristics of generated two-part counterpoints. The rules acted as comments about the generated music from a critic. The musical agent would amend its policy based on these comments. In our approach, each episode was a complete 32-bar two-part counterpoint. Form and other contexts (such as chordal context) were incorporated into the system via the critic’s rules and the usage of context dependent Q-tables. In this approach the behaviours could be easily varied by amending the critic’s rules and the contexts. We provide the details of the proposed approach and sample results, as well as discuss further research.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Allan, M., Williams, C.K.: Harmonising chorales by probabilistic inference. In: Saul, L., Weiss, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems, vol. 17. MIT Press, Cambridge
Chen, C.C.J., Miikkulainen, R.: Creating melodies with evolving recurrent neural networks. In: Proceedings of the 2001 International Joint Conference on Neural Network, IJCNN 2001, Washington DC. IEEE, Los Alamitos (2001)
Cont, A., Dubnov, S., Assayag, G.: Anticipatory model of musical style imitation using collaborative and competitive reinforcement learning. In: Butz, M.V., Sigaud, O., Pezzulo, G., Baldassarre, G. (eds.) ABiALS 2006. LNCS (LNAI), vol. 4520, pp. 285–306. Springer, Heidelberg (2007)
Collins, N.: Reinforcement learning for live musical agents. In: Proceedings of the International Computer Music Conference, ICMC 2008, Belfast, Ireland, August 24-29 (2008)
Ebcioglu, K.: An expert system for harmonizing four-part chorales. In: Balaban, M., Ebcioglu, K., Laske, O. (eds.) Understanding Music with AI: Perspectives on music cognition, Ch.12, pp. 294–333. The AAAI Press/The MIT Press
Franklin, J.A., Manfredi, V.U.: Nonlinear credit assignment for musical sequences. In: Second International Workshop on Intelligent System Design and Application, pp. 245–250 (2002)
Horner, A., Goldberg, D.E.: Genetic algorithms and computer-assisted music composition. In: Belew, R., Booker, L. (eds.) Proceedings of the Fourth International Conference on Genetic Algorithms. Morgan Kauffman, San Francisco (1991)
Kennedy, M.: The Concise Oxford Dictionary of Music. Oxford University Press, Oxford (1996)
Phon-Amnuaisuk, S.: Control language for harmonisation process. In: Anagnostopoulou, C., Ferrand, M., Smaill, A. (eds.) ICMAI 2002. LNCS (LNAI), vol. 2445, p. 155. Springer, Heidelberg (2002)
Saul, L.K., Jordan, M.I.: Mixed memory markov models: Decomposing complex stochastic processes as mixtures of simpler ones. Machine Learning 37(1), 75–87 (1999)
Schultz, W.: Predictive reward signal of dopamine neurons. Journal of Neurophysiology 80, 1–27 (1998)
Sutton, R.S.: Generalization in reinforcement learning: Successful examples using sparse course coding. In: Touretzky, D.S., Mozer, M.C., Hasselmo, M.E. (eds.) Proceedings of the Advances in Neural Information Processing Systems Proceedings, pp. 1038–1044. MIT Press, Cambridge (1996)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. The MIT Press, A Bradford Book (1998)
Taylor, E.: The AB Guide to Music Theory (part I and part II). The Associated Board of the Royal Schools of Music (1989)
Todd, P.M., Werner, G.M.: Frankensteinian methods for evolutionary music composition. In: Griffith, N., Todd, P.M. (eds.) Musical Networks: Parallel Distributed Perception and Performance, pp. 313–340. The MIT Press, Cambridge
Toiviainen, P., Eerola, T.: A method for comparative analysis of folk music based on musical feature extraction and neural networks. In: VII International Symposium on Systematic and Comparative Musicology and III International Conference on Cognitive Musicology, University of Jyvskyl, Finland, August 16-19 (2001)
Watkins, C.J., Dayan, P.: Q-learning Machine. Learning 8, 279–292 (1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Phon-Amnuaisuk, S. (2009). Generating Tonal Counterpoint Using Reinforcement Learning. In: Leung, C.S., Lee, M., Chan, J.H. (eds) Neural Information Processing. ICONIP 2009. Lecture Notes in Computer Science, vol 5863. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-10677-4_66
Download citation
DOI: https://doi.org/10.1007/978-3-642-10677-4_66
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-10676-7
Online ISBN: 978-3-642-10677-4
eBook Packages: Computer ScienceComputer Science (R0)