Skip to main content
SpringerLink
Log in

Algorithmic approach to sonification of classical Chinese poetry

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The classical Chinese poetry is a remarkable form of art in traditional Chinese character. However, it is difficult for people who are unfamiliar with ancient Chinese to experience the artistic content of the poetry. In this study, a sonification scheme, Tx2Ms (Text-to-Music), is proposed to extract the poetry features between lines in verses; moreover, dynamics and interval relations are modeled to map those features to the movement of multi-dimensional musical elements such as durations. This conversion is based on poetry intonation and acoustic analysis of the pronunciations of poems; and a stochastic compositional algorithm is created by applying a Markov chain. In addition, the best pentatonic mode for a specific poem is recommended according to the formants analysis. Therefore, the sonification of classical Chinese poetry not only provides a novel way for people to appreciate Chinese poetry but also enriches the state of mind and imagery in the delivery process, and the experiment results show that the proposed system is successfully accepted by most people.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

References

  1. Bain R (2006) The AIMS Project: creative experiments in musical sonification. International Computer Music Conference ICMC 2006, New Orleans, Louisiana, USA (November)

  2. Ballas A (1994) Delivery of information through sound. In: Kramer G (ed) Auditory display: sonification, audification and auditory interfaces. Addison-Wesley, Reading, pp 79–94

    Google Scholar 

  3. Barrass S, Kramer G (1999) Using sonification. Multimedia systems 7(1) http://www.springerlink.com/content/xd19ftjpfe3pb30l/fulltext.pdf (accessed 3 April 2011)

  4. Blattner M, Papp L, Glinert P (1994) Sonic enhancement of two-dimensional graphics displays. In: Kramer G (ed) Auditory display: sonification, audification and auditory interfaces. Addison-Wesley, Reading, pp 447–470

    Google Scholar 

  5. Brewster A, Wright C, Edwards N (1993) An evaluation of earcons for use in auditory human-computer interfaces. In: Ashlund S, Mullet K, Henderson A, Hollnagel E, White T (eds) Proceedings of InterCHI’93. ACM, Amsterdam, pp 222–227

    Google Scholar 

  6. Chadabe J (1997) Electric sound: the past and promise of electronic. Music Prentice Hall, pp 178, ISBN 9780133032314

  7. Chen H-H (2004) Introduction to composition in Chinese Poetry-Tsu. Wu-Nan Book Co., Taipei

    Google Scholar 

  8. Clement J (1998) Learning harmonic progression using Markov models. http://www-lrn.cs.umass.edu/lab-lunch/papers/clement98learning.pdf (accessed 10 May 2009)

  9. Coagula software website: http://hem.passagen.se/rasmuse/Coagula.htm (accessed 8 December 2010)

  10. Cope D (1992) Computer modeling of musical intelligence in experiments in musical intelligence. Comp Music J 16(2):69–83

    Article  Google Scholar 

  11. Cope D (2004) virtual music: computer synthesis of musical style. MIT Press

  12. Doornbusch P (2002) A brief survey of mapping in algorithmic composition. International Computer Music Conference, Göteborg

    Google Scholar 

  13. Durate J, Hsiao S-C, Huang C-F, and Winsor P (2006) The applications of Sieve Theory in Algorithmic Composition using MAX/MSP and BASIC. The 2nd International Conference WOCMAT, Workshop for Computer Music and Audio Technology, Taipei, Taiwan, pp 96–99 (March)

  14. Fagerlönn J, Liljedahl M (2009) Awesome sound design tool: a web based utility that invites end users into the audio design process. In Proceedings of the 15th International Conference on Auditory Display (ICAD2009), Copenhagen, Denmark (May)

  15. Farbood M, Schoner B (2001) Analysis and synthesis of palestrina-style counterpoint using Markov chains. International Computer Music Conference, Havana, Cuba, pp 18–22 (September)

  16. Fowler A (1986) Franz Liszt’s Petrarch Sonnets: the persistent poetic problem. Indiana Theor Rev 7(2):48–68

    Google Scholar 

  17. Franz M (1998) Markov chains as tools for jazz improvisation analysis. Master thesis, Virginia Polytechnic Institute and State University

  18. Garzonis S, Jones S, Jay T, O’Neill E (2009) Auditory icon and earcon mobile service notifications: intuitiveness, learnability, memorability and preference. Proceedings of the 27th International Conference on Human Factors in Computing Systems, CHI 2009,ACM, Boston, MA, USA, 4–9 April 2009

  19. Gaser C, Schlaug G (2003) Brain structures differ between musicians and non-musicians. J Neurosci 23(27):9240–9245 (October)

    Google Scholar 

  20. Goina M, Polotti P (2008) Elementary gestalts for gesture sonification. Proceedings of the 2008 International Conference on New Interfaces for Musical Expression (NIME–08). Genova, Italy, pp 150–153 (June)

  21. Hiller L, Isaacson L (1959) Experimental music. McGraw-Hill

  22. Hsu C-Y (1997) Theory on classical Chinese poetry composition. Hungyeh Publishing, Taipei

    Google Scholar 

  23. Hunter P, Schellenberg E, Schimmack U (2008) Mixed affective responses to music with conflicting cues. Cogn Emotion 22(2):327–352

    Article  Google Scholar 

  24. Hussein K, Tilevich E, Bukvic I, Kim S (2009) Sonification design guidelines to enhance program comprehension. In Proceedings of ICPC‘2009, pp 120–129

  25. Image-to-Sound Mapping website: http://www.seeingwithsound.com/im2sound.htm (accessed 8 December 2010)

  26. Jeon M, Davison B, Nees M, Wilson J, Walker B (2009) Enhanced auditory menu cues improve dual task performance and are preferred with in-vehicle technologies. In Proceedings of the 1st International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI ‘09). ACM, New York, NY, USA, pp 91–98

  27. Kaper G, Wiebel E, Tipei S (1999) Data Sonification and Sound Visualization. Computing in Science and Engineering 1, no. 4, http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=774840&isnumber=16814 (accessed 9 May 2009)

  28. Kramer G (1994) Some organizing principles for representing data with sound. In: Kramer G (ed) Auditory display: sonification, audification and auditory interfaces. Addison-Wesley, Reading, pp 185–221

    Google Scholar 

  29. Kramer G (1994) An introduction to auditory display. In: Kramer G (ed) Auditory display: sonification, audification and auditory interfaces. Addison-Wesley, Reading, pp 1–77

    Google Scholar 

  30. Kramer G et al (1999) Sonification report: status of the field and research agenda. Report prepared for the National Science Foundation by members of the International Community for Auditory Display. http://www.icad.org/websiteV2.0/References/nsf.html (accessed 3 April 2011)

  31. Langston P (1989) Six techniques for algorithmic music composition. 15th International Computer Music Conference (ICMC) in Columbus, Ohio (November)

  32. Lerdahl F (2001) The sounds of poetry viewed as music. Ann N Y Acad Sci 337–354 (June)

  33. McAlpine K, Miranda E, Hoggar S (1999) Making music with algorithms: a case-study system. Comp Music J 23(2):19–30

    Article  Google Scholar 

  34. McCormack J (1996) Grammar Based Music Composition. In: Stocker R et al (eds) Complex systems 96: from local interactions to global phenomena. ISO, Amsterdam, pp 320–336

    Google Scholar 

  35. Mitchell D (1985) Gustav Mahler: songs and symphonies of life and death: interpretations and annotations. University of California Press (November)

  36. Moore R (1990) Chapter five: composing. In Elements of computer music. Prentice-Hall, Englewood Cliffs

  37. Rahn J (1980) Basic atonal theory. New York: Longman; London and Toronto: Prentice Hall International, ISBN 0-02-873160-3. Reprinted 1987, New York: Schirmer Books; London: Collier Macmillan

  38. Rothstein J (1995) MIDI: A Comprehensive Introduction, Computer Music and Digital Audio Series, Vol.7, A-R Editions, Inc.

  39. Saue S (2000) A model for interaction in exploratory sonification displays. In Proceedings of the International Conference on Auditory Display (ICAD2000), in Atlanta, Georgia (April)

  40. Scaletti C (1994) Sound synthesis algorithms for auditory data representations. In: Kramer G (ed) Auditory display: sonification, audification and auditory interfaces. Addison-Wesley, Reading, pp 223–251

    Google Scholar 

  41. Shan K, Chiu S-C (2010) Algorithmic compositions based on discovered musical patterns. J Multimed Tools Appl 46:1–23

    Article  Google Scholar 

  42. Spiliotopoulos D, Stavropoulou P, Kouroupetroglou G (2009) Acoustic rendering of data tables using earcons and prosody for document accessibility. In: Stephanidis C (ed) Proceedings of the 5th International Conference on Universal Access in Human-Computer Interaction. Part III: Applications and Services (UAHCI ‘09). Springer, Berlin, pp 587–596

    Chapter  Google Scholar 

  43. Sturm L (2001) Synthesis and algorithmic composition techniques derived from particle physics. Proc. 8th Biennial Arts Tech. Symp., Connecticut College, New London

  44. Vazquez-Alvarez Y, Oakley I, and Brewster A (2010) Urban sound gardens: supporting overlapping audio landmarks in exploratory environments. In Proceedings of multimodal location based techniques for extreme navigation workshop, Pervasive, Helsinki, Finland, pp 37–38

  45. Verbeurgt K, Dinolfo M, and Fayer M (2004) Extracting patterns in music for composition via Markov chains. Innovations in Applied Artificial Intelligence. 17th International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert Systems, IEA/AIE 2004 Ottawa, Canada, pp 1123–1132 (May)

  46. Verron C, Aramaki M, Kronland-Martinet R, Pallone G (2009) Analysis/synthesis and spatialization of noisy environmental sounds. In Proceedings of the 15th International Conference on Auditory Display (ICAD2009), Copenhagen, Denmark, pp 36–40 (May)

  47. Walker N, Cothran T (2003) Sonification sandbox: a graphical toolkit for auditory graphs. In Proceedings of the International Conference on Auditory Display (ICAD2003), Boston, USA (July)

  48. Walker B, Kogan A (2009) Spearcon performance and preference for auditory menus on a mobile phone. In: Stephanidis C (ed) Proceedings of the 5th International on ConferenceUniversal Access in Human-Computer Interaction. Part II: Intelligent and Ubiquitous Interaction Environments (UAHCI ‘09). Springer, Berlin, pp 445–454

    Google Scholar 

  49. Winsor P (1989) The computer composer’s toolbox. Windcrest Books, Blue Ridge Summit

    Google Scholar 

  50. Winsor P (1990) Computer music in C. Blue Ridge Summit, Penn. Windcrest Books

  51. Winsor P (1992) Automated music composition. University of North Texas Press, Denton

    Google Scholar 

  52. Wu X, Li Z-N (2008) A study of image-based music composition. 2008 IEEE International Conference on Multimedia and Expo, ICME 2008 Proceedings, Burnaby, BC, pp 1345–1348

  53. Yang C-C (2004) A study of constructing chinese poetry knowledge base—Su-Shi’s classical poetry knowledge base system. Master thesis, National Chiao Tung University

Download references

Acknowledgement

The authors would like to appreciate the support from National Science Council projects of Taiwan: NSC99-2410-H-155-035-MY2.

Author information

Authors and Affiliations

  1. Department of Information Communication, Yuan Ze University, 135 Yuan-Tung Road, Chung-Li, Taiwan, 32003, Republic of China

    Chih-Fang Huang

  2. Department of Mechanical Engineering, National Chiao-Tung University, 1001 University Road,, Hsinchu, Taiwan, 300, Republic of China

    Hsiang-Pin Lu

  3. Institute of Music, National Chiao-Tung University, 1001 University Road, Hsinchu, Taiwan, 300, Republic of China

    Jenny Ren

Authors
  1. Chih-Fang Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hsiang-Pin Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jenny Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chih-Fang Huang.

Appendix

Appendix

The following is a translation of the poems exemplified in this paper, excerpted from the book, Chinese-English Readings in poems of Tang dynasty and tsu of Sung dynasty (中英對照讀唐詩宋詞), written by Shih, Ying-Chou (施穎洲).

figure e

Rights and permissions

Reprints and permissions

About this article

Cite this article

Huang, CF., Lu, HP. & Ren, J. Algorithmic approach to sonification of classical Chinese poetry. Multimed Tools Appl 61, 489–518 (2012). https://doi.org/10.1007/s11042-011-0856-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-011-0856-4

Keywords

Search

Navigation