Abstract
Creating music can be challenging for children aged 4–6. Making the creation procedure more interesting has been proved to stimulate and build children’s interests in music. Based on the design concept of synesthesia from vision to hearing, we propose an application by which children aged 4–6 can create collages by matching different images of scenes and animals. The information contained in the created collages is then used as input parameters for generating music so as to create melodies corresponding to the collages. To evaluate our approach, we conducted a quantitative and qualitative user study with children (n = 6) by using our initial prototype. We concluded that the proposed application helps cultivate children's willingness to create melodies, and helps children better perceive and understand music by combining scenes and animal images in the collages.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ostrov, J.M., Gentile, D.A., Mullins, A.D.: Evaluating the effect of educational media exposure on aggression in early childhood. J. Appl. Dev. Psychol. 34(1), 38–44 (2013)
Taylor, H.: Music with the Under-Fours by Susan Young. Routledge Falmer, London and New York (2003). (139 p. £ 14.99, paperback. Br. J. Music Educ. 21(1), 131 (2004))
McKinnon, I.: Children’s music journey: the development of an interactive software solution for early childhood music education. Comput. Entertain. 3(4), 1–10 (2005)
Oestermeier, U., et al.: LEGO music: learning composition with bricks. In: Proceedings of the 14th International Conference on Interaction Design and Children, pp. 283–286. Association for Computing Machinery, Boston, Massachusetts (2015)
Tidjani, A., Cho, E., Lee, P.: MuSme: a tangible skin suit for music creation. In: Proceedings of the TEI 2016: Tenth International Conference on Tangible, Embedded, and Embodied Interaction, pp. 743–748. Association for Computing Machinery, Eindhoven, Netherlands (2016)
Ling, Y.: Interactive visual music education software for children. In: Proceedings of the 20th International Academic Mindtrek Conference, pp. 453–456. Association for Computing Machinery, Tampere, Finland (2016)
Moruzzi, C.: Creative AI: music composition programs as an extension of the composer’s mind. In: Müller, V.C. (ed.) PT-AI 2017. SAPERE, vol. 44, pp. 69–72. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-96448-5_8
Brown, L.L.: The benefits of music education. PBS KIDS for Parents (2012)
Jaschke, A.C., et al.: Music education and its effect on intellectual abilities in children: a systematic review. Rev. Neurosci. 24(6), 665–675 (2013)
Ouyang, L., et al.: Designing tangible interactions with children for pre-school music education. In: 32nd Australian Conference on Human-Computer Interaction, pp. 687–691. Association for Computing Machinery Sydney, NSW, Australia (2020)
Altieri, S., et al.: Music education technology for older elementary school aged children: playing around with mini Amadeus Jr. J. Comput. Sci. Coll. 20(5), 102–103 (2005)
He, L., et al.: TempoString: a tangible tool for children's music creation. In: Proceedings of the 2012 ACM Conference on Ubiquitous Computing, pp. 643–644. Association for Computing Machinery, Pittsburgh, Pennsylvania (2012)
Rigler, J., Seldess, Z.: The music creator: an interactive system for musical exploration and education. In: Proceedings of the 7th International Conference on New Interfaces for Musical Expression, pp. 415–416. Association for Computing Machinery, New York (2007)
Akiyama, Y., Oore, S.: Place and play: a digital tool for children to create and record music. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 735–738. Association for Computing Machinery, Florence, Italy (2008)
Voong, M., Beale, R.: Music organisation using colour synaesthesia. In: CHI 2007 Extended Abstracts on Human Factors in Computing Systems, pp. 1869–1874. Association for Computing Machinery San Jose, CA, USA (2007)
Jin, S., Qin, J., Li, W.: Draw portraits by music: a music based image style transformation. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 4399–4400. Association for Computing Machinery, Seattle, WA, USA (2020)
Šimbelis, V.V., Lundström, A.: Synthesis in the audiovisual. In: Proceedings of the CHI 2016 Conference Extended Abstracts on Human Factors in Computing Systems, pp. 301–304. Association for Computing Machinery, San Jose, California, USA (2016)
Kritikos, D., Karpouzis, K.: From pixels to notes: a computational implementation of synaesthesia for cultural artefacts. arXiv preprint: arXiv:2101.12038 (2021)
Frid, E., Gomes, C., Jin, Z.: Music creation by example. In: Proceedings of the 2020 CHI 2020 Conference on Human Factors in Computing Systems, pp. 1–13. Association for Computing Machinery, Honolulu, HI, USA (2020)
Louie, R., et al.: Novice-AI music co-creation via AI-steering tools for deep generative models. In: Proceedings of the CHI 2020 Conference on Human Factors in Computing Systems, pp. 1–13. Association for Computing Machinery, Honolulu, HI, USA (2020)
Ning, C., Zhou, S.: The music pattern: a creative tabletop music creation platform. Comput. Entertain. 8(2), 13 (2011)
Kunigk, D.: Generate-music (2020). https://github.com/kiecodes/generate-music
Acknowledgments
This research was funded by the Engineering Research Center of Computer Aided Product Innovation Design, Ministry of Education, Fundamental Research Funds for the Central Universities, National Natural Science Foundation of China (Grant No. 52075478), and National Social Science Foundation of China (Grant No. 21AZD056).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yan, G., Yao, C., Zhang, C., Wang, J., Hu, Y., Ying, F. (2022). MusicCollage: A Music Composition Tool for Children Based on Synesthesia and a Genetic Algorithm. In: Streitz, N.A., Konomi, S. (eds) Distributed, Ambient and Pervasive Interactions. Smart Living, Learning, Well-being and Health, Art and Creativity. HCII 2022. Lecture Notes in Computer Science, vol 13326. Springer, Cham. https://doi.org/10.1007/978-3-031-05431-0_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-05431-0_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-05430-3
Online ISBN: 978-3-031-05431-0
eBook Packages: Computer ScienceComputer Science (R0)