Abstract
Methods of sonification based on the design and control of sound synthesis is presented in this paper. The semiotics of isolated sounds was evidenced by performing fundamental studies using a combined acoustical and brain imaging (event-related potentials) approach. The perceptual cues (which are known as invariants) responsible for the evocations elicited by the sounds generated by impacts, moving sound sources, dynamic events and vehicles (car-door closing and car engine noise) were then identified based on physical and perceptual considerations. Lastly, some examples of the high-level control of a synthesis process simulating immersive 3-D auditory scenes, interacting objects and evoked dynamics are presented.



Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
Cycling ’74, http://www.cycling74.com/downloads/.
References
Aramaki M, Besson M, Kronland-Martinet R, Ystad S (2009) Timbre perception of sounds from impacted materials: behavioral, electrophysiological and acoustic approaches. In: Ystad S, Kronland-Martinet R, Jensen K (eds) Computer music modeling and retrieval—genesis of meaning of sound and music, LNCS, vol 5493. pp 1–17 Springer, Berlin
Aramaki M, Besson M, Kronland-Martinet R, Ystad S (2011) Controlling the perceived material in an impact sound synthesizer. IEEE Trans Audio Speech Lang Process 19(2):301–314
Aramaki M, Gondre C, Kronland-Martinet R, Voinier T, Ystad S (2010a) Imagine the sounds: an intuitive control of an impact sound synthesizer. In: Ystad S, Aramaki M, Kronland-Martinet R, Jensen J (eds) Auditory display, lecture notes in computer science, vol 5954. pp 408–421 Springer, Berlin
Aramaki M, Marie C, Kronland-Martinet R, Ystad S, Besson M (2010b) Sound categorization and conceptual priming for nonlinguistic and linguistic sounds. J Cogn Neurosci 22(11):2555–2569
Ballas JA (1993) Common factors in the identification of an assortment of brief everyday sounds. J Exp Psychol Hum Percept Perform 19(2):250–267
Barthet M, Depalle P, Kronland-Martinet R, Ystad S (2010) Acoustical correlates of timbre and expressiveness in clarinet performance. Music Percept Interdiscip J 28(2):135–154
Barthet M, Depalle P, Kronland-Martinet R, Ystad S (2011) Analysis-by-synthesis of timbre, timing, and dynamics in expressive clarinet performance. Music Percept Interdiscip J 28(3):265–278
Bezat MC, Roussarie V, Kronland-Martinet R, Ystad S, McAdams S (2006) Perceptual analyses of action-related impact sounds. In: Proceedings of the 6th European conference on noise control euronoise 2006. Tampere, Finland
Bezat MC, Roussarie V, Voinier T, Kronland-Martinet R, Ystad S (2007) Car door closure sounds: characterization of perceptual properties through analysis-synthesis approach. In: Proceedings of the 19th international congress on acoustics. Madrid, Spain
Chowning JM (1971) The simulation of moving sound sources. J Audio Eng Soc 19(1):2–6
Gaver WW (1993a) How do we hear in the world? Explorations of ecological acoustics. Ecol Psychol 5(4):285–313
Gaver WW (1993b) What in the world do we hear? An ecological approach to auditory source perception. Ecol Psychol 5(1):1–29
Gibson JJ (1986) The ecological approach to visual perception. Lawrence Erlbaum Associates, Hillsdale
Gygi B, Kidd GR, Watson CS (2007) Similarity and categorization of environmental sounds. Percept Psychophys 69(6):839–855
Gygi B, Shafiro V (2007) General functions and specific applications of environmental sound research. Front Biosci 12:3152–3166
Huang NE, Shen Z, Long S, Wu M, Shih H, Zheng Q, Yen N, Tung C, Liu HH (1998) The empirical mode decomposition and hilbert spectrum for nonlinear and non-stationary time series analysis. Proc Roy Soc Lond A 454:903–995
Kronland-Martinet R, Voinier T (2008) Real-time perceptual simulation of moving sources: application to the leslie cabinet and 3d sound immersion. EURASIP J Audio Speech Music Process. doi:10.1155/2008/849696
Kutas M, Hillyard SA (1980) Reading senseless sentences: brain potentials reflect semantic incongruity. Sci Agric 207:203–204
McAdams S, Bigand E (1993) Thinking in sound: the cognitive psychology of human audition. Oxford University Press, Oxford
Merer A, Ystad S, Kronland-Martinet R, Aramaki M (2008) Semiotics of sounds evoking motions: categorization and acoustic features. In: Kronland-Martinet R, Ystad S, Jensen K (eds) Computer music modeling and retrieval—sense of sounds, lecture notes in computer science, vol 4969. pp 139–158 Springer, Berlin
Merer A, Ystad S, Kronland-Martinet R, Aramaki M (2010) On the potentiality of abstract sounds in perception research. In: Proceedings of the 7th international symposium on computer music modeling and retrieval, CMMR 2010—music is in the sound
Pressnitzer D, Gnansia D (2005) Real time auditory model. In: Proceedings of international computer music conference. Barcelona, Spain
Rosenblum LD, Carello C, Pastore RE (1987) Relative effectiveness of three stimulus variables for locating a moving sound source. Perception 16(2):175–186
Rugg MD, Coles MGH (1995) Electrophysiology of mind. Event-related brain potentials and cognition, chap. The ERP and cognitive psychology: conceptual issues, No. 25 in Oxford psychology. Oxford University Press, Oxford, pp 27–39
Schaeffer P (1966) Traité des objets musicaux. Ed. du Seuil
Schön D, Ystad S, Kronland-Martinet R, Besson M (2010) The evocative power of sounds: conceptual priming between words and nonverbal sounds. J Cogn Neurosci 22(5):1026–1035
Sciabica JF, Bezat MC, Roussarie V, Kronland–Martinet R, Ystad S (2010) Towards timbre modeling of sounds inside accelerating cars. In: Ystad S, Aramaki M, Kronland-Martinet R, Jensen J (eds) Auditory display, lecture notes in computer science, vol 5954. pp 377–392 Springer, Berlin
Terhardt E, Stoll G, Seewann M (1982) Pitch of complex signals according to virtual-pitch theory: tests, examples, and predictions. J Acoust Soc Am 71:671–678
Väljamäe A, Larsson P, Västfjäll D, Kleiner M (2005) Travelling without moving: auditory scene cues for translational self-motion. In: Proceedings of the 11th international conference on auditory display (ICAD ’05)
Vanderveer NJ (1979) Ecological acoustics: human perception of environmental sounds. Ph.D. thesis, Georgia Institute of Technology
Verron C, Aramaki M, Kronland-Martinet R, Pallone G (2010) A 3d immersive synthesizer for environmental sounds. IEEE Trans Audio Speech Lang Process 18(6):1550–1561
Verron C, Pallone G, Aramaki M, Kronland-Martinet R (2009) Controlling a spatialized environmental sound synthesizer. In: Proceedings of the IEEE workshop on applications of signal processing to audio and acoustics (WASPAA). New Paltz, NY. 18–21 October 2009, pp 321–324
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kronland-Martinet, R., Ystad, S. & Aramaki, M. High-level control of sound synthesis for sonification processes. AI & Soc 27, 245–255 (2012). https://doi.org/10.1007/s00146-011-0340-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00146-011-0340-8