Abstract
Building an intelligent system for automatically composing music like human beings has been actively investigated during the last decade. In this work, we propose a new approach for automatically creating hymns by training a variational attention model from a large collection of religious songs. We compare our method with two other techniques by using Seq2Seq and attention models and measure the corresponding performance by BLEU-N scores, the entropy, and the edit distance. Experimental results show that the proposed method can achieve a promising performance that is able to give an additional contribution to the current study of music formulation. Finally, we publish our dataset online for further research related to the problem.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. CoRR abs/1409.0473, September 2014. arXiv e-prints https://arxiv.org/abs/1409.0473
Bahuleyan, H., Mou, L., Vechtomova, O., Poupart, P.: Variational attention for sequence-to-sequence models. CoRR abs/1712.08207 (2017)
Brunner, G., Wattenhofer, R., Wiesendanger, J.: JamBot: music theory aware chord based generation of polyphonic with LSTMs. CoRR (2017)
Choi, K., Fazekas, G., Sandler, M.B.: Text-based LSTM networks for automatic music composition. CoRR (2016)
Coffman, D.: Measuring musical originality using information theory. Psychol. Music 20(2), 154–161 (1992). https://doi.org/10.1177/0305735692202005
Eck, D., Schmidhuber, J.: A first look at music composition using LSTM recurrent neural networks. Technical report no. IDSIA-07-02. IDSIA/USI-SUPSI, Instituto Dalle Molle di studi sull’ intelligenza artificiale, Manno, Switzerland (2002). http://www.iro.umontreal.ca/~eckdoug/blues/IDSIA-07-02.pdf
Gregor, K., Danihelka, I., Wierstra, D.: DRAW: a recurrent neural network for image generation. CoRR (2015)
Hiller, L.A., Isaacson, L.M.: Experimental Music; Composition with an Electronic Computer. Greenwood Publishing Group Inc., Westport (1979). ISBN: 0313221588. https://dl.acm.org/citation.cfm?id=578548
Hochreiter, S., Bengio, Y., Frasconi, P., Schmidhuber, J.: Gradient flow in recurrent nets: the difficulty of learning long-term dependencies (2001)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Jordan, M.I.: Artificial neural networks. In: Jordan, M.I. (ed.) Attractor Dynamics and Parallelism in a Connectionist Sequential Machine, pp. 112–127. IEEE Press, Piscataway (1990)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014). http://arxiv.org/abs/1412.6980
Luong, T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1412–1421 (2015)
Mozer, M.C.: Neural network music composition by prediction: exploring the benefits of psychoacoustic constraints and multi-scale processing. Connect. Sci. 6, 247–280 (1994)
Navarro, G.: A guided tour to approximate string matching. ACM Comput. Surv. 33(1), 31–88 (2001). https://doi.org/10.1145/375360.375365
Nguyen, B.: A public dataset for hymn generation (2019). https://sites.google.com/site/ntbinhpolytechnique/datasets
Lewis, J.P.: Algorithms for music composition by neural nets: improved CBR paradigms. In: Proceeding of the International Computer Music Conference (1989)
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL 2002, pp. 311–318 (2002)
Pudaruth, S., Amourdon, S., Anseline, J.: Automated generation of song lyrics using CFGS. In: The Seventh International Conference on Contemporary Computing (IC3) (2014)
Schoenberg, A.: Theory of Harmony. University of California Press, California (1983)
Shi, D.: A study on neural network language modeling. CoRR abs/1708.07252 (2017). http://arxiv.org/abs/1708.07252
Sturm, B.L., Santos, J.F., Ben-Tal, O., Korshunova, I.: Music transcription modelling and composition using deep learning. CoRR abs/1604.08723 (2016)
Sutskever, I., Martens, J., Hinton, G.: Generating text with recurrent neural networks. In: Proceedings of the 28th International Conference on International Conference on Machine Learning, pp. 1017–1024. Omnipress (2011)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. CoRR abs/1409.3215 (2014)
Tomczak, M.: BachBot. Master Thesis (2016)
Yi, L., Goldsmith, J.: Automatic generation of four-part harmony. In: Proceedings of the Fifth UAI Conference on Bayesian Modeling Applications Workshop, vol. 268 (2007)
Zeyer, A., Doetsch, P., Voigtlaender, P., Schluter, R., Ney, H.: A comprehensive study of deep bidirectional LSTM RNNS for acoustic modeling in speech recognition. In: ICASSP 2017, pp. 2462–2466 (2017)
Zhou, C., Neubig, G.: Morphological inflection generation with multi-space variational encoder-decoders. In: SIGMORPHON 2017, pp. 58–65 (2017)
Acknowledgments
CKH and BTN would like to thank The National Foundation for Science and Technology Development (NAFOSTED), University of Science, and Inspectorio Research Lab in Viet Nam for supporting two authors throughout this paper.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Cao, H.K., Ly, D.T., Nguyen, D.M., Nguyen, B.T. (2019). Automatically Generate Hymns Using Variational Attention Models. In: Lu, H., Tang, H., Wang, Z. (eds) Advances in Neural Networks – ISNN 2019. ISNN 2019. Lecture Notes in Computer Science(), vol 11555. Springer, Cham. https://doi.org/10.1007/978-3-030-22808-8_32
Download citation
DOI: https://doi.org/10.1007/978-3-030-22808-8_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-22807-1
Online ISBN: 978-3-030-22808-8
eBook Packages: Computer ScienceComputer Science (R0)