Skip to main content
SpringerLink
Log in
Book cover

Multimedia Data Mining and Analytics pp 419–446Cite as

Listen to the Sound of Data

  • Chapter
  • First Online:

Abstract

Timestamped observations, generally known as time series, may contain valuable information about a variety of natural and man-made phenomena ranging from weather changes to stock markets. Our capability to collect such data has increased dramatically due to advances in computing and sensory technologies. Visualization is known as a very effective tool for interactive data exploration tasks. In this research, we have tested the hypothesis that musical sonification (the use of musical audio) can serve as a viable alternative to visualization of time-series data whenever the visual representation is unavailable or impossible to use. We have developed a time-series sonification technique, which utilizes some important features of Western tonal music to convert univariate and multivariate time series into a musical equivalent. The technique was used to conduct two online user studies, where the subjects were asked questions about the data behavior by listening to a musical display of time series rather than viewing their visual representation. The results of both studies indicate that our methodology for musical representation of time-dependent data allows most users, including people with low musical hearing ability, to successfully perform a variety of common data exploration tasks.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    MSQ format was designed by Siegfried Koepf and Bernd Haerpfer in 1998. The idea is to have a platform-independent, easily readable, and editable file format qualified for algorithmic manipulation and composition as well as for real-time controlling MIDI instruments. More details about the MSQ Project are available at [17].

  2. 2.

    Pitch is related to the repetition rate of the waveform of a sound; for a pure tone this corresponds to its frequency.

References

  1. Barras S, Kramer G (1999) Using sonification. Multimed Syst 7(1):23–31

    Article  Google Scholar 

  2. Brockwell PJ, Davis RA (2002) Time series: theory and methods, 2nd edn. Springer, New York

    Google Scholar 

  3. El-Azm F (2005) Sonification and augmented data sets in binary classification. PhD Dissertation, Institute of Informatics and Mathematical Modeling at the Technical University of Denmark

    Google Scholar 

  4. Gaver WW (1994) Using and creating auditory icons. In: Kramer G (ed) Auditory display: sonification, audification, and auditory interfaces. Addison-Wesley, Reading, pp 417–446

    Google Scholar 

  5. Hermann T (2008) Taxonomy and definitions for sonification and auditory display. In: Susini P, Warusfel O (eds) Proceedings of the 14th international conference on auditory display (ICAD 2008). IRCAM, Paris

    Google Scholar 

  6. Hermann T, Hunt A (2005) An introduction to interactive sonification. IEEE Multimed 12(2):20–24

    Article  Google Scholar 

  7. Hermann T, Ritter H (1999) Listen to your data: Model-based sonification for data analysis. In: Proceedings of the international symposium on intelligent multimedia and distance education (ISIMADE’99), Baden-Baden

    Google Scholar 

  8. Keogh E, Kasetty S (2003) On the need for time series data mining benchmarks: a survey and empirical demonstration. Data Min Knowl Discov 7(4):349–371

    Article  MathSciNet  Google Scholar 

  9. Keogh E, Chu S, Hart D, Pazzani M (2004) Segmenting time series: a survey and novel approach. In: Last M, Kandel A, Bunke H (eds) Data mining in time series databases. World Scientific Publishing Company, Singapore, pp 1–21

    Chapter  Google Scholar 

  10. Kramer G (1994) An introduction to auditory display. In: Kramer G (ed) Auditory display: sonification, audification and auditory interfaces, vol. XVIII. Addison-Wesley, pp 1–78

    Google Scholar 

  11. Kramer G, Walker B, Bonebright T, Cook P, Flowers J, Miner N, Neuhoff J (1999) Sonification report: status of the field and research agenda. Technical report, ICAD

    Google Scholar 

  12. Last M, Gorelik A (2008) Using sonification for mining time series data. In: Proceedings of the 9th international. Workshop on Multimedia Data Mining (MDM/KDD 2008), Las Vegas. 24 August 2008, pp 63–72

    Google Scholar 

  13. Last M, Kandel A, Bunke H (2004) Data mining in time series databases. In: Machine Perception and Artificial Intelligence, vol. 57. World Scientific, Singapore

    Google Scholar 

  14. Last M, Klein Y, Kandel A (2001) Knowledge discovery in time series databases. IEEE Trans Syst Man Cybern, Part B—Cybern, 31(1): 160–169

    Google Scholar 

  15. Leumi Group. http://www.bankleumi.co.il

  16. Liu L-M, Bhattacharyya S, Sclove SL, Chena R, Lattyak WJ (2001) Data mining on time series: an illustration using fast-food restaurant franchise data. Comput Stat Data Anal 37(4):455–476

    Article  MATH  Google Scholar 

  17. MSQ Project, http://www.aconnect.de/friends/editions/computer/msq2/msq.html

  18. Muller W, Schumann H (2003) Visualization methods for time-dependent data—an overview. In: Proceedings of the 2003 winter simulation conference, vol 1, pp 737–745

    Google Scholar 

  19. Nesbitt KV, Barrass S (2004) Finding trading patterns in stock market data. IEEE Comput Graph Appl 24(5):45–55

    Article  Google Scholar 

  20. Noirhomme-Fraiture M, Schöller O, Demoulin C, Simoff S (2002) Sonification of time dependent data. In: Proceedings of international workshop on visual data mining. Helsinki, pp 113–125

    Google Scholar 

  21. Patterson, R (1982) Guidelines for auditory warning systems on civil aircraft. Civil aviation authority

    Google Scholar 

  22. Pauletto S, Hunt A (2009) Interactive sonification of complex data. Int J Hum-Comput Stud 67(11):923–933

    Article  Google Scholar 

  23. Pauletto S, Hunt A (2004) A toolkit for interactive sonification. In: Proceedings of the international conference of auditory display (ICAD). Sydney

    Google Scholar 

  24. Peretz I, Zatorre RJ (2005) Brain organization for music processing. Annu Rev Psychol 56:89–114

    Article  Google Scholar 

  25. Sonification examples, http://www.ise.bgu.ac.il/faculty/mlast/data/Midi and Pics.zip

  26. Tel-Aviv Stock Exchange, http://www.tase.co.il

  27. UCR Time series classification/clustering page, http://www.cs.ucr.edu/~eamonn/time_series_data/

  28. Walker BN, Godfrey MT, Orlosky JE, Bruce C, Sanford J (2006) Aquarium sonification: soundscapes for accessible dynamic informal learning environments. In: Proceedings of the international conference on auditory display (ICAD2006). London, pp 238–241

    Google Scholar 

  29. Walker BN, Kim J, Pendse A (2007) Musical soundscapes for an accessible aquarium: Bringing dynamic exhibits to the visually impaired. In: Proceedings of the international computer music conference (ICMC 2007). Denmark

    Google Scholar 

  30. Walker BN, Lindsay J, Nance A, Nakano Y, Palladino DK, Dingler T, Jeon M (2013) Spearcons (speech-based earcons) improve navigation performance in advanced auditory menus. Hum Factors: J Hum Factors Ergon Soc 55(1):157–182

    Article  Google Scholar 

  31. Williamson J, Murray-Smith, R (2002) Audio feedback with gesture recognition. Technical report TR-2002-127, Department of Computer Science, University of Glasgow

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Systems Engineering, Ben-Gurion University of the Negev, Marcus Family Campus, 84105, Beersheva, Israel

    Mark Last & Anna Usyskin (Gorelik)

Authors
  1. Mark Last
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anna Usyskin (Gorelik)
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mark Last .

Editor information

Editors and Affiliations

  1. IBM Corp., Durham, North Carolina, USA

    Aaron K. Baughman

  2. Nokia Inc., Sunnyvale, California, USA

    Jiang Gao

  3. Google Inc., Mountain View, California, USA

    Jia-Yu Pan

  4. 4i, Inc., Carlsbad, California, USA

    Valery A. Petrushin

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Last, M., Usyskin (Gorelik), A. (2015). Listen to the Sound of Data. In: Baughman, A., Gao, J., Pan, JY., Petrushin, V. (eds) Multimedia Data Mining and Analytics. Springer, Cham. https://doi.org/10.1007/978-3-319-14998-1_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-14998-1_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-14997-4

  • Online ISBN: 978-3-319-14998-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation