Skip to main content
SpringerLink
Log in

Navigation of interactive sonifications and visualisations of time-series data using multi-touch computing

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

This paper discusses interaction design for interactive sonification and visualisation of data in multi-touch contexts. Interaction design for data analysis is becoming increasingly important as data becomes more openly available. We discuss how navigation issues such as zooming, selection, arrangement and playback of data relate to both the auditory and visual modality in different ways, and how they may be linked through the modality of touch and gestural interaction. For this purpose we introduce a user interface for exploring and interacting with representations of time-series data simultaneously in both the visual and auditory modalities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Adcock M, Barrass S (2004) Cultivating design patterns for auditory displays. In: Proceedings of the 10th international conference on auditory display, Sydney

    Google Scholar 

  2. Burkhard R et al. (2007) Visualisation summit 2007: Ten research goals for 2010. Inform Visual 6(3):169–188

    Article  MathSciNet  Google Scholar 

  3. Aldersey-Williams H, Hall P, Sargent T, Antonelli P (2008) Design and the elastic mind. The Museum of Modern Art, New York

    Google Scholar 

  4. Anderson E (1935) The irises of the gaspe peninsula. Bull Am Iris Soc 59:2–5

    Google Scholar 

  5. Barrass S (2003) Sonification design patterns. In: Proceedings of the 9th international conference on auditory display, Boston

    Google Scholar 

  6. Barrass S, Schaffert N, Barrass T (2010) Probing preferences between six designs of interactive sonifications for recreational sports, health and fitness. In: Proceedings of ISon 2010, 3rd interactive sonification workshop, Stockholm, Sweden

    Google Scholar 

  7. Beilharz K, Ferguson S (2009) An interface design for aesthetic and interactive sonification. In: Proceedings of the 15th international conference on auditory display, Copenhagen, Denmark

    Google Scholar 

  8. Bertelsen O, Pold S (2004) Criticism as an approach to interface aesthetics. In: Nordic conference on human-computer interaction, Tampere, Finland

    Google Scholar 

  9. Bovermann T, Groten J, de Campo A, Eckel G (2007) Juggling sounds. In: International workshop on interactive sonification, York, UK

    Google Scholar 

  10. Bovermann T, Hermann T, Ritter H (2006) Tangible data scanning sonification model. In: Proceedings of the 12th international conference on auditory display, London, UK

    Google Scholar 

  11. Chen C (2005) Top 10 unsolved information visualisation problems. IEEE Comput Graph Appl 25(4):12–16

    Article  Google Scholar 

  12. Collins A, Apted T, Kay J (2007) Tabletop file system access: associative and hierarchical approaches. In: IEEE international workshop on horizontal interactive human-computer systems (TABLETOP’07), pp 113–120

    Chapter  Google Scholar 

  13. Collins A, Kay J (2008) Collaborative personal information management with shared, interactive tabletops. In: CHI 2008, Florence, Italy

    Google Scholar 

  14. Cowie A (2010) Collaborative concept mapping at an interactive tabletop. PhD thesis

  15. Ferguson S (2009) Exploratory sound analysis: statistical sonifications for the investigation of sound. PhD thesis

  16. Flowers JH, Buhman DC, Turnage KD (1997) Cross-modal equivalence of visual and auditory scatterplots for exploring bivariate data samples. Hum Factors 39(3):341–351

    Article  Google Scholar 

  17. Fry B (2004) Computational information design. PhD thesis

  18. Fry B (2008) Nature hapmap cover. Nature 437(7063):1 (cover)

    Google Scholar 

  19. Geiselhart K (2010) Towards global governance—interactive technologies and global accountability measures. Public Commun Rev 1(1):37–46

    Google Scholar 

  20. Hermann T (2002) Sonification for exploratory data analysis. PhD thesis

  21. Hermann T (2008) Taxonomy and definitions for sonification and auditory display. In: Proceedings of the 2008 international conference on auditory display, Paris, France

    Google Scholar 

  22. Hermann T, Bovermann T, Riedenklau E, Ritter H (2007) Tangible computing for interactive sonification of multivariate data. In: International workshop on interactive sonification, York, UK

    Google Scholar 

  23. Hermann T, Niehus C, Ritter H (2003) Interactive visualisation and sonification for monitoring complex processes. In: Proceedings of the 9th international conference on auditory display, Boston, USA

    Google Scholar 

  24. Hochenbaum J, Vallis O (2009) Bricktable: a musical tangible multi-touch interface. In: Proceedings of Berlin open conference 09, Berlin, Germany

    Google Scholar 

  25. Hochenbaum J, Vallis O, Diakopoulos D, Murphy J, Kapur A (2010) Designing expressive musical interfaces for tabletop surfaces. In: Proceedings of the 2010 international conference on new interfaces for musical expression, Sydney, Australia

    Google Scholar 

  26. Hunt A, Hermann T (2004) The importance of interaction in sonification. In: Proceedings of the 10th international conference on auditory display, Sydney, Australia

    Google Scholar 

  27. Judelman G (2004) Aesthetics and inspiration for visualisation design: Bridging the gap between art and science. In: International conference on information visualisation (IV’04), London, England

    Google Scholar 

  28. Kosara R (2007) Visualisation criticism—the missing link between information visualisation and art. In: International conference on information visualisation (IV’07), Zurich, Switzerland

    Google Scholar 

  29. Kramer G (1993) Auditory display: sonification, audification, and auditory interfaces, Perseus, New York

    Google Scholar 

  30. Kramer G, Walker BN, Bonebright T, Cook PR, Flowers JH, Miner N, Neuhoff JG (1997) Sonification report: status of the field and research agenda. Tech rep, National Science Foundation

  31. Mayo E, Steinberg T (2010) The power of information: an independent review

  32. Nees M, Walker B (2006) Relative intensity of auditory context for auditory graph design. In: 12th international conference on auditory display, London, UK, pp 95–98

    Google Scholar 

  33. Pousman Z, Stasko J, Mateas M (2007) Casual information visualisation: depictions of data in everyday life. IEEE Trans Vis Comput Graph 13(6):1145–1152

    Article  Google Scholar 

  34. Sack W (2006) Aesthetics of information visualisation. In: Paul C, Vesna V, Lovejoy M (eds) Context providers. MIT Press, Boston

    Google Scholar 

  35. Scha R, Bod R (1993) Computational esthetics. Inf Infbeleid 11(1):54–63

    Google Scholar 

  36. Shneiderman B (1996) The eyes have it: a task by data type taxonomy for information visualisations. In: IEEE symposium on visual languages, Boulder, CO, USA

    Google Scholar 

  37. Song H, Beilharz K (2007) Concurrent auditory stream discrimination in auditory graphing. J Comput 3(1):79–87

    Google Scholar 

  38. Tufte ER (1992) Envisioning information. Graphics Press, Cheshire

    Google Scholar 

  39. Tünnermann R, Hermann T (2009) Multi-touch interactions for model-based sonification. In: Proceedings of the 15th international conference on auditory display, Copenhagen, Denmark

    Google Scholar 

  40. Moere Vande A (2005) Form follows data: the symbiosis between design and information visualisation. In: International conference on computer-aided architectural design (CAADfutures’05). OKK Verlag, Vienna

    Google Scholar 

  41. Vesna V (2007) Seeing the world in a grain of sand: the database aesthetics of everything. In: Vesna V (ed) Database aesthetics—art in the age of information overflow, electronic mediations. University of Minnesota Press, Minneapolis

    Google Scholar 

  42. Viegas F, Wattenberg M (2007) Artistic data visualisation: beyond visual analytic. In: Online communities and social computing. Lecture notes in computer science, vol 4564(15). Springer, Berlin, pp 182–191

    Chapter  Google Scholar 

  43. Walker BN, Nees MA (2005) An agenda for research and development of multimodal graphs. In: Proceedings of the 11th international conference on auditory display, Limerick, Ireland

    Google Scholar 

  44. Wilkinson L (2005) The grammar of graphics, 2nd edn. Springer, Berlin

    MATH  Google Scholar 

  45. Zacks J, Levy E, Tverksy B, Schiano D (1998) Reading bar graphs: effects of extraneous depth cues and graphical context. J Exp Psychol, Appl 4(2):119–138

    Article  Google Scholar 

  46. Zacks J, Levy E, Tversky B, Schiano D (2001) Graphs in print. In: Olivier P, Anderson M, Meyer B (eds) Diagrammatic representation and reasoning. Springer, London

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Design, Architecture and Building, University of Technology, Sydney, Australia

    Sam Ferguson, Kirsty Beilharz & Claudia A. Calò

Authors
  1. Sam Ferguson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kirsty Beilharz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Claudia A. Calò
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sam Ferguson.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ferguson, S., Beilharz, K. & Calò, C.A. Navigation of interactive sonifications and visualisations of time-series data using multi-touch computing. J Multimodal User Interfaces 5, 97–109 (2012). https://doi.org/10.1007/s12193-011-0075-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-011-0075-3

Keywords

Search

Navigation