Skip to main content
SpringerLink
Log in

High-level control of sound synthesis for sonification processes

  • Original Article
  • Published:
AI & SOCIETY Aims and scope Submit manuscript

Abstract

Methods of sonification based on the design and control of sound synthesis is presented in this paper. The semiotics of isolated sounds was evidenced by performing fundamental studies using a combined acoustical and brain imaging (event-related potentials) approach. The perceptual cues (which are known as invariants) responsible for the evocations elicited by the sounds generated by impacts, moving sound sources, dynamic events and vehicles (car-door closing and car engine noise) were then identified based on physical and perceptual considerations. Lastly, some examples of the high-level control of a synthesis process simulating immersive 3-D auditory scenes, interacting objects and evoked dynamics are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. Cycling ’74, http://www.cycling74.com/downloads/.

References

  • Aramaki M, Besson M, Kronland-Martinet R, Ystad S (2009) Timbre perception of sounds from impacted materials: behavioral, electrophysiological and acoustic approaches. In: Ystad S, Kronland-Martinet R, Jensen K (eds) Computer music modeling and retrieval—genesis of meaning of sound and music, LNCS, vol 5493. pp 1–17 Springer, Berlin

    Chapter  Google Scholar 

  • Aramaki M, Besson M, Kronland-Martinet R, Ystad S (2011) Controlling the perceived material in an impact sound synthesizer. IEEE Trans Audio Speech Lang Process 19(2):301–314

    Article  Google Scholar 

  • Aramaki M, Gondre C, Kronland-Martinet R, Voinier T, Ystad S (2010a) Imagine the sounds: an intuitive control of an impact sound synthesizer. In: Ystad S, Aramaki M, Kronland-Martinet R, Jensen J (eds) Auditory display, lecture notes in computer science, vol 5954. pp 408–421 Springer, Berlin

  • Aramaki M, Marie C, Kronland-Martinet R, Ystad S, Besson M (2010b) Sound categorization and conceptual priming for nonlinguistic and linguistic sounds. J Cogn Neurosci 22(11):2555–2569

    Article  Google Scholar 

  • Ballas JA (1993) Common factors in the identification of an assortment of brief everyday sounds. J Exp Psychol Hum Percept Perform 19(2):250–267

    Article  Google Scholar 

  • Barthet M, Depalle P, Kronland-Martinet R, Ystad S (2010) Acoustical correlates of timbre and expressiveness in clarinet performance. Music Percept Interdiscip J 28(2):135–154

    Google Scholar 

  • Barthet M, Depalle P, Kronland-Martinet R, Ystad S (2011) Analysis-by-synthesis of timbre, timing, and dynamics in expressive clarinet performance. Music Percept Interdiscip J 28(3):265–278

    Google Scholar 

  • Bezat MC, Roussarie V, Kronland-Martinet R, Ystad S, McAdams S (2006) Perceptual analyses of action-related impact sounds. In: Proceedings of the 6th European conference on noise control euronoise 2006. Tampere, Finland

  • Bezat MC, Roussarie V, Voinier T, Kronland-Martinet R, Ystad S (2007) Car door closure sounds: characterization of perceptual properties through analysis-synthesis approach. In: Proceedings of the 19th international congress on acoustics. Madrid, Spain

  • Chowning JM (1971) The simulation of moving sound sources. J Audio Eng Soc 19(1):2–6

    Google Scholar 

  • Gaver WW (1993a) How do we hear in the world? Explorations of ecological acoustics. Ecol Psychol 5(4):285–313

    Article  MathSciNet  Google Scholar 

  • Gaver WW (1993b) What in the world do we hear? An ecological approach to auditory source perception. Ecol Psychol 5(1):1–29

    Article  MathSciNet  Google Scholar 

  • Gibson JJ (1986) The ecological approach to visual perception. Lawrence Erlbaum Associates, Hillsdale

    Google Scholar 

  • Gygi B, Kidd GR, Watson CS (2007) Similarity and categorization of environmental sounds. Percept Psychophys 69(6):839–855

    Article  Google Scholar 

  • Gygi B, Shafiro V (2007) General functions and specific applications of environmental sound research. Front Biosci 12:3152–3166

    Article  Google Scholar 

  • Huang NE, Shen Z, Long S, Wu M, Shih H, Zheng Q, Yen N, Tung C, Liu HH (1998) The empirical mode decomposition and hilbert spectrum for nonlinear and non-stationary time series analysis. Proc Roy Soc Lond A 454:903–995

    Article  MathSciNet  MATH  Google Scholar 

  • Kronland-Martinet R, Voinier T (2008) Real-time perceptual simulation of moving sources: application to the leslie cabinet and 3d sound immersion. EURASIP J Audio Speech Music Process. doi:10.1155/2008/849696

  • Kutas M, Hillyard SA (1980) Reading senseless sentences: brain potentials reflect semantic incongruity. Sci Agric 207:203–204

    Google Scholar 

  • McAdams S, Bigand E (1993) Thinking in sound: the cognitive psychology of human audition. Oxford University Press, Oxford

    Book  Google Scholar 

  • Merer A, Ystad S, Kronland-Martinet R, Aramaki M (2008) Semiotics of sounds evoking motions: categorization and acoustic features. In: Kronland-Martinet R, Ystad S, Jensen K (eds) Computer music modeling and retrieval—sense of sounds, lecture notes in computer science, vol 4969. pp 139–158 Springer, Berlin

    Chapter  Google Scholar 

  • Merer A, Ystad S, Kronland-Martinet R, Aramaki M (2010) On the potentiality of abstract sounds in perception research. In: Proceedings of the 7th international symposium on computer music modeling and retrieval, CMMR 2010—music is in the sound

  • Pressnitzer D, Gnansia D (2005) Real time auditory model. In: Proceedings of international computer music conference. Barcelona, Spain

  • Rosenblum LD, Carello C, Pastore RE (1987) Relative effectiveness of three stimulus variables for locating a moving sound source. Perception 16(2):175–186

    Article  Google Scholar 

  • Rugg MD, Coles MGH (1995) Electrophysiology of mind. Event-related brain potentials and cognition, chap. The ERP and cognitive psychology: conceptual issues, No. 25 in Oxford psychology. Oxford University Press, Oxford, pp 27–39

    Google Scholar 

  • Schaeffer P (1966) Traité des objets musicaux. Ed. du Seuil

  • Schön D, Ystad S, Kronland-Martinet R, Besson M (2010) The evocative power of sounds: conceptual priming between words and nonverbal sounds. J Cogn Neurosci 22(5):1026–1035

    Article  Google Scholar 

  • Sciabica JF, Bezat MC, Roussarie V, Kronland–Martinet R, Ystad S (2010) Towards timbre modeling of sounds inside accelerating cars. In: Ystad S, Aramaki M, Kronland-Martinet R, Jensen J (eds) Auditory display, lecture notes in computer science, vol 5954. pp 377–392 Springer, Berlin

  • Terhardt E, Stoll G, Seewann M (1982) Pitch of complex signals according to virtual-pitch theory: tests, examples, and predictions. J Acoust Soc Am 71:671–678

    Article  Google Scholar 

  • Väljamäe A, Larsson P, Västfjäll D, Kleiner M (2005) Travelling without moving: auditory scene cues for translational self-motion. In: Proceedings of the 11th international conference on auditory display (ICAD ’05)

  • Vanderveer NJ (1979) Ecological acoustics: human perception of environmental sounds. Ph.D. thesis, Georgia Institute of Technology

  • Verron C, Aramaki M, Kronland-Martinet R, Pallone G (2010) A 3d immersive synthesizer for environmental sounds. IEEE Trans Audio Speech Lang Process 18(6):1550–1561

    Article  Google Scholar 

  • Verron C, Pallone G, Aramaki M, Kronland-Martinet R (2009) Controlling a spatialized environmental sound synthesizer. In: Proceedings of the IEEE workshop on applications of signal processing to audio and acoustics (WASPAA). New Paltz, NY. 18–21 October 2009, pp 321–324

Download references

Author information

Authors and Affiliations

  1. Laboratoire de Mécanique et d’Acoustique, CNRS, 31, Chemin Joseph Aiguier, 13402, Marseille Cedex 20, France

    Richard Kronland-Martinet & Sølvi Ystad

  2. Institut de Neurosciences Cognitives de la Méditerranée, CNRS, 31, Chemin Joseph Aiguier, 13402, Marseille Cedex 20, France

    Mitsuko Aramaki

  3. Aix-Marseille University, 38, Boulevard Charles Livon, Marseille, France

    Mitsuko Aramaki

Authors
  1. Richard Kronland-Martinet
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sølvi Ystad
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mitsuko Aramaki
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Richard Kronland-Martinet.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kronland-Martinet, R., Ystad, S. & Aramaki, M. High-level control of sound synthesis for sonification processes. AI & Soc 27, 245–255 (2012). https://doi.org/10.1007/s00146-011-0340-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00146-011-0340-8

Keywords

Search

Navigation