Exploring Classical Music Narratives Through Multimodality in AR/VR Experiences

Abstract

Although Music is considered the field of emotions and moods, every composition has a structure: beginning, development, climax and conclusion. In classical music, there are many genres, but each piece always tells a story. Even more, most classical compositions follow the familiar structure of Aristotelian drama. Classical music narrative can take a number of forms: as music analysis on structures of the form (e.g., Sonata form with contrasting themes), emotional narrative (usually miniatures dedicated to one mood range, e.g., the genre of Prelude) or a program narrative according to composer’s notes (e.g., Berlioz’s Symphonie Fantastique). Generally speaking, perception of music is multisensory, so when music tells a story, it does so in a multimodal fashion. In this position paper, we present our investigation into visualisation of music narrative based on different forms of music narratives, and then show how visuals developed by cross-modal associations and synaesthesia art can be used to construct music narrative that is interactive. Our ultimate aim is to develop the approach into AR/VR visualisations of classical music with a strong narrative content.