Skip to main content
SpringerLink
Log in

Using Crossmodal Correspondence Between Colors and Music to Enhance Online Art Exhibition Visitors’ Experience

  • Conference paper
  • First Online:
Information for a Better World: Normality, Virtuality, Physicality, Inclusivity (iConference 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13971))

Included in the following conference series:

Abstract

Background music has been widely used in online art exhibitions to enhance visitors’ art appreciation experience. At present, what music to use is highly dependent on exhibition designers’ personal understanding, experience, or even intuition. In fact, it is possible to systematically identify matching music for given artwork due to the crossmodal correspondence between visual and auditory inputs. This paper presents an exploratory study of the crossmodal correspondence between color hues and music tempos and its effects on online art appreciation experience based on two experiments. According to Experiment 1, warm colors were congruent with fast music, whereas cool colors were congruent with slow music. In Experiment 2, congruent and incongruent background music was used in an online exhibition of oil paintings, and several findings were engendered. For warm-colored paintings, congruent (fast) music helped visitors remembered significant more paintings. For cool-colored paintings, visitors collected or shared more paintings when hearing congruent (slow) music but viewed the paintings for longer durations when hearing incongruent (fast) music. The findings not only enrich the understanding of the roles of audio-visual crossmodal correspondence, but also inform online art exhibition designers of how to make background music work in harmony with artwork to better engage visitors.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Vi, C.T., Ablart, D., Gatti, E., et al.: Not just seeing, but also feeling art: mid-air haptic experiences integrated in a multisensory art exhibition. Int. J. Hum Comput Stud. 108, 1–14 (2017)

    Article  Google Scholar 

  2. Spence, C.: Scenting the anosmic cube: on the use of ambient scent in the context of the art gallery or museum. i-Perception 11(6), 1–26 (2020)

    Google Scholar 

  3. Guo, K., Fan, A., Lehto, X., et al.: Immersive digital tourism: the role of multisensory cues in digital museum experiences. J. Hosp. Tour. Res., 1–23 (2021)

    Google Scholar 

  4. Chen, C.-L., Tsai, C.-G.: The influence of background music on the visitor museum experience: a case study of the Laiho memorial museum. Taiwan. Visit. Stud. 18(2), 183–195 (2015)

    Article  Google Scholar 

  5. Ausín, J.M., Bigne, E., Marín, J., et al.: The background music-content congruence of TV advertisements: a neurophysiological study. Eur. Res. Manag. Bus. Econ. 27(2), 100154 (2021)

    Article  Google Scholar 

  6. Kim, K.-H., Iwamiya, S.-I.: Formal congruency between telop patterns and sound effects. Music Percept. 25(5), 429–448 (2008)

    Article  Google Scholar 

  7. Yu, C.-E., Xie, S.Y., Wen, J.: Coloring the destination: The role of color psychology on Instagram. Tour. Manag. 80, 104110 (2020)

    Article  Google Scholar 

  8. Edworthy, J., Waring, H.: The effects of music tempo and loudness level on treadmill exercise. Ergonomics 49(15), 1597–1610 (2006)

    Article  Google Scholar 

  9. Spence, C.: Crossmodal correspondences: a tutorial review. Atten. Percept. Psychophys. 73(4), 971–995 (2011)

    Article  Google Scholar 

  10. Lin, A., Scheller, M., Feng, F., et al.: Feeling colours: crossmodal correspondences between tangible 3D objects, colours and emotions. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (2021)

    Google Scholar 

  11. Sun, X., Li, X., Ji, L., et al.: An extended research of crossmodal correspondence between color and sound in psychology and cognitive ergonomics. PeerJ 6, e4443 (2018)

    Article  Google Scholar 

  12. Jonas, C., Spiller, M.J., Hibbard, P.: Summation of visual attributes in auditory–visual crossmodal correspondences. Psychon. Bull. Rev. 24(4), 1104–1112 (2017). https://doi.org/10.3758/s13423-016-1215-2

    Article  Google Scholar 

  13. Motoki, K., Velasco, C.: Taste-shape correspondences in context. Food Qual. Prefer. 88, 104082 (2021)

    Article  Google Scholar 

  14. Hagtvedt, H., Brasel, S.A.: Cross-modal communication: sound frequency influences consumer responses to color lightness. J. Mark. Res. 53(4), 551–562 (2016)

    Article  Google Scholar 

  15. Metatla, O., Correia, N.N., Martin, F., et al.: Tap the shapetones: Exploring the effects of crossmodal congruence in an audio-visual interface. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, New York, NY. ACM Press (2016)

    Google Scholar 

  16. Klapetek, A., Ngo, M.K., Spence, C.: Does crossmodal correspondence modulate the facilitatory effect of auditory cues on visual search? Atten. Percept. Psychophys. 74(6), 1154–1167 (2012). https://doi.org/10.3758/s13414-012-0317-9

    Article  Google Scholar 

  17. Velasco, C., Woods, A.T., Petit, O., et al.: Crossmodal correspondences between taste and shape, and their implications for product packaging: a review. Food Qual. Prefer. 52, 17–26 (2016)

    Article  Google Scholar 

  18. Spence, C.: Olfactory-colour crossmodal correspondences in art, science, and design. Cognit. Res. Princ. Implic. 5(1), 1–21 (2020). https://doi.org/10.1186/s41235-020-00246-1

    Article  Google Scholar 

  19. Cirrincione, A., Estes, Z., Carù, A.: The effect of ambient scent on the experience of art: not as good as it smells. Psychol. Mark. 31(8), 615–627 (2014)

    Article  Google Scholar 

  20. Andersson, P.K., Kristensson, P., Wästlund, E., et al.: Let the music play or not: the influence of background music on consumer behavior. J. Retail. Consum. Serv. 19(6), 553–560 (2012)

    Article  Google Scholar 

  21. Kang, H.J., Williamson, V.J.: Background music can aid second language learning. Psychol. Music 42(5), 728–747 (2014)

    Article  Google Scholar 

  22. Flowers, P.J., O’Neill, A.A.M.: Self-reported distractions of middle school students in listening to music and prose. J. Res. Music Educ. 53(4), 308–321 (2005)

    Article  Google Scholar 

  23. Webb, R.C.: Music, mood, and museums: a review of the consumer literature on background music. Visit. Stud. 8(2), 15–29 (1996)

    Article  Google Scholar 

  24. Tan, S.-L., Cohen, A.J., Lipscomb, S.D., et al.: The Psychology of Music in Multimedia. Oxford University Press, Oxford (2013)

    Google Scholar 

  25. North, A.C., Hargreaves, D.J., McKendrick, J.: The influence of in-store music on wine selections. J. Appl. Psychol. 84(2), 271 (1999)

    Article  Google Scholar 

  26. Wu, C.-S., Cheng, F.-F., Yen, D.C.: The atmospheric factors of online storefront environment design: an empirical experiment in Taiwan. Inf. Manag. 45(7), 493–498 (2008)

    Article  Google Scholar 

  27. Cheng, F.-F., Wu, C.-S., Yen, D.C.: The effect of online store atmosphere on consumer’s emotional responses–an experimental study of music and colour. Behav. Inf. Technol. 28(4), 323–334 (2009)

    Article  Google Scholar 

  28. Qi, Y., Huang, F., Li, Z., et al.: Crossmodal correspondences in the sounds of Chinese instruments. Perception 49(1), 81–97 (2020)

    Article  Google Scholar 

  29. Anikin, A., Johansson, N.: Implicit associations between individual properties of color and sound. Atten. Percept. Psychophys. 81(3), 764–777 (2018). https://doi.org/10.3758/s13414-018-01639-7

    Article  Google Scholar 

  30. Adeli, M., Rouat, J., Molotchnikoff, S.: Audiovisual correspondence between musical timbre and visual shapes. Front. Hum. Neurosci. 8, 352 (2014)

    Article  Google Scholar 

  31. Palmer, S.E., Schloss, K.B., Xu, Z., et al.: Music–color associations are mediated by emotion. Proc. Natl. Acad. Sci. 110(22), 8836–8841 (2013)

    Article  Google Scholar 

  32. Whiteford, K.L., Schloss, K.B., Helwig, N.E., et al.: Color, music, and emotion: bach to the blues. i-Perception 9(6), 1–27 (2018)

    Google Scholar 

  33. Chiou, R., Rich, A.N.: Cross-modality correspondence between pitch and spatial location modulates attentional orienting. Perception 41(3), 339–353 (2012)

    Article  Google Scholar 

  34. Walker, P., Bremner, J.G., Mason, U., et al.: Preverbal infants’ sensitivity to synaesthetic cross-modality correspondences. Psychol. Sci. 21(1), 21–25 (2010)

    Article  Google Scholar 

  35. Fernández-Prieto, I., Navarra, J., Pons, F.: How big is this sound? Crossmodal association between pitch and size in infants. Infant Behav. Dev. 38, 77–81 (2015)

    Article  Google Scholar 

  36. Bookbinder, S., Brainerd, C.: Emotionally negative pictures enhance gist memory. Emotion 17(1), 102 (2017)

    Article  Google Scholar 

  37. McKnight, P.E., Najab, J.: Mann‐Whitney U test. The Corsini encyclopedia of psychology, 1 (2010)

    Google Scholar 

  38. McCrum-Gardner, E.: Which is the correct statistical test to use? Br. J. Oral Maxillofac. Surg. 46(1), 38–41 (2008)

    Article  Google Scholar 

  39. Wichmann, F.A., Sharpe, L.T., Gegenfurtner, K.R.: The contributions of color to recognition memory for natural scenes. J. Exp. Psychol. Learn. Mem. Cognit. 28(3), 509 (2002)

    Article  Google Scholar 

  40. Lee, K., Joshi, K.: Examining the use of status quo bias perspective in IS research: need for re-conceptualizing and incorporating biases. Inf. Syst. J. 27(6), 733–752 (2017)

    Article  Google Scholar 

  41. Bedford, L.: The Art of Museum Exhibitions: How Story and Imagination Create Aesthetic Experiences. Routledge (2016)

    Google Scholar 

  42. North, A.C., Hargreaves, D.J., Krause, A.E.: Music and Consumer Behaviour. Oxford University Press, Oxford (2009)

    Google Scholar 

  43. Daniel, K.: Thinking, Fast and Slow. Macmillan, New York (2011)

    Google Scholar 

  44. North, A.C., Sheridan, L.P., Areni, C.S.: Music congruity effects on product memory, perception, and choice. J. Retail. 92(1), 83–95 (2016)

    Article  Google Scholar 

  45. Ranasinghe, N., Jain, P., Thi Ngoc Tram, N., et al.: Season traveller: multisensory narration for enhancing the virtual reality experience. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (2018)

    Google Scholar 

  46. Halabi, O., Saleh, M.: Augmented reality flavor: cross-modal mapping across gustation, olfaction, and vision. Multimedia Tools Appl. 80(30), 36423–36441 (2021). https://doi.org/10.1007/s11042-021-11321-0

    Article  Google Scholar 

Download references

Acknowledgement

This research has been made possible through the financial support of the National Social Science Foundation of China under Grant No. 22&ZD325 and the National Natural Science Foundation of China under Grant No. 72074173.

Author information

Authors and Affiliations

  1. School of Information Management, Wuhan University, Wuhan, Hubei, China

    Qian Guo & Tingting Jiang

  2. Center for Studies of Information Resources, Wuhan University, Wuhan, Hubei, China

    Tingting Jiang

Authors
  1. Qian Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tingting Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tingting Jiang .

Editor information

Editors and Affiliations

  1. iSchool Organization, Berlin, Germany

    Isaac Sserwanga

  2. Victoria University of Wellington, Wellington, New Zealand

    Anne Goulding

  3. University of Missouri, Chicago, IL, USA

    Heather Moulaison-Sandy

  4. University of South Australia, Adelaide, SA, Australia

    Jia Tina Du

  5. University of Porto, Porto, Portugal

    António Lucas Soares

  6. Monash University, Clayton, VIC, Australia

    Viviane Hessami

  7. University of Tennessee at Knoxville, Knoxville, TN, USA

    Rebecca D. Frank

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Guo, Q., Jiang, T. (2023). Using Crossmodal Correspondence Between Colors and Music to Enhance Online Art Exhibition Visitors’ Experience. In: Sserwanga, I., et al. Information for a Better World: Normality, Virtuality, Physicality, Inclusivity. iConference 2023. Lecture Notes in Computer Science, vol 13971. Springer, Cham. https://doi.org/10.1007/978-3-031-28035-1_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-28035-1_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-28034-4

  • Online ISBN: 978-3-031-28035-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation