Erica Volta
ABSTRACT
Musical communication can be considered as a form of social interaction and it requires a certain degree of inter-individual cortical synchronization. Scientific literature has shown that visual experience is necessary for social interaction and visual impairment negatively influences nonverbal communication both in children and adults. In this study, we present a pilot multimodal dataset to investigate the role that visual experience plays in the context of musical interaction. We selected a mixed interaction scenario (i.e., sighted and blind performers) to investigate non-verbal communication patterns. In particular, we recorded motion tracking data to analyze the interaction between two musicians in three different contexts with the same cellist as soloist accompanied by a (1) blind pianist, (2) sighted pianist, and (3) sighted but blindfolded pianist. Recordings comprised upper-body motion capture and audio data. We also investigated human perception of changes in non-verbal behaviors, by means of an online annotation questionnaire. Preliminary results are presented.
- Jean-Julien Aucouturier and Clément Canonne. 2017. Musical friends and foes: The social cognition of affiliation and control in improvised interactions. Cognition 161(2017), 94–108.Google Scholar
Cross Ref
- Claudio Babiloni, Fabrizio Vecchio, Francesco Infarinato, Paola Buffo, Nicola Marzano, Danilo Spada, Simone Rossi, Ivo Bruni, Paolo M Rossini, and Daniela Perani. 2011. Simultaneous recording of electroencephalographic data in musicians playing in ensemble. cortex 47, 9 (2011), 1082–1090.Google Scholar
- Leonardo Badino, Alessandro D’Ausilio, Donald Glowinski, Antonio Camurri, and Luciano Fadiga. 2014. Sensorimotor communication in professional quartets. Neuropsychologia 55(2014), 98–104.Google ScholarCross Ref
- Laura Bishop, Carlos Cancino-Chacón, and Werner Goebl. 2019. Moving to communicate, moving to interact: Patterns of body motion in musical duo performance. Music Perception: An Interdisciplinary Journal 37, 1 (2019), 1–25.Google ScholarCross Ref
- Laura Bishop and Werner Goebl. 2015. When they listen and when they watch: Pianists’ use of nonverbal audio and visual cues during duet performance. Musicae Scientiae 19, 1 (2015), 84–110.Google ScholarCross Ref
- Antonio Camurri, Paolo Coletta, Alberto Massari, Barbara Mazzarino, Massimiliano Peri, Matteo Ricchetti, Andrea Ricci, and Gualtiero Volpe. 2004. Toward real-time multimodal processing: EyesWeb 4.0. In Proceedings of the AISB 2004 Convention: Motion, Emotion and Cognition. 22–26.Google Scholar
- Lee J Cronbach. 1951. Coefficient alpha and the internal structure of tests. psychometrika 16, 3 (1951), 297–334.Google Scholar
- Alessandro D’Ausilio, Leonardo Badino, Yi Li, Sera Tokay, Laila Craighero, Rosario Canto, Yiannis Aloimonos, and Luciano Fadiga. 2012. Leadership in orchestra emerges from the causal relationships of movement kinematics. PLoS one 7, 5 (2012), e35757.Google ScholarCross Ref
- Thomas Dolk, Roman Liepelt, Wolfgang Prinz, and Katja Fiehler. 2013. Visual experience determines the use of external reference frames in joint action control. PLoS One 8, 3 (2013), e59008.Google ScholarCross Ref
- Alessandro D’Ausilio, Giacomo Novembre, Luciano Fadiga, and Peter E Keller. 2015. What can music tell us about social interaction?Trends in cognitive sciences 19, 3 (2015), 111–114.Google Scholar
- Joseph L Fleiss and Jacob Cohen. 1973. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and psychological measurement 33, 3 (1973), 613–619.Google Scholar
- Melissa J Frame. 2000. The relationship between visual impairment and gestures. Journal of Visual Impairment & Blindness 94, 3 (2000), 155–171.Google ScholarCross Ref
- Donald Glowinski, Fabrizio Bracco, Carlo Chiorri, and Didier Grandjean. 2016. Music ensemble as a resilient system. Managing the unexpected through group interaction. Frontiers in Psychology 7 (2016), 1548.Google ScholarCross Ref
- Donald Glowinski, Maurizio Mancini, Roddy Cowie, Antonio Camurri, Carlo Chiorri, and Cian Doherty. 2013. The movements made by performers in a skilled quartet: a distinctive pattern, and the function that it serves. Frontiers in psychology 4 (2013), 841.Google Scholar
- Jana M Iverson and Susan Goldin-Meadow. 2001. The resilience of gesture in talk: Gesture in blind speakers and listeners. Developmental Science 4, 4 (2001), 416–422.Google ScholarCross Ref
- Satoshi Kawase. 2014. Gazing behavior and coordination during piano duo performance. Attention, Perception, & Psychophysics 76, 2 (2014), 527–540.Google ScholarCross Ref
- J Richard Landis and Gary G Koch. 1977. An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics (1977), 363–374.Google Scholar
- Jacques Launay. 2015. Musical sounds, motor resonance, and detectable agency. Empirical musicology review: EMR 10, 1-2 (2015), 30.Google ScholarCross Ref
- Janeen D Loehr, Dimitrios Kourtis, Cordula Vesper, Natalie Sebanz, and Günther Knoblich. 2013. Monitoring individual and joint action outcomes in duet music performance. Journal of cognitive neuroscience 25, 7 (2013), 1049–1061.Google ScholarDigital Library
- Nikki Moran, Lauren V Hadley, Maria Bader, and Peter E Keller. 2015. Perception of ‘back-channeling’nonverbal feedback in musical duo improvisation. PLoS One 10, 6 (2015), e0130070.Google ScholarCross Ref
- Johanna Sänger, Viktor Müller, and Ulman Lindenberger. 2012. Intra-and interbrain synchronization and network properties when playing guitar in duets. Frontiers in human neuroscience 6 (2012), 312.Google Scholar
- Johanna Sänger, Viktor Müller, and Ulman Lindenberger. 2013. Directionality in hyperbrain networks discriminates between leaders and followers in guitar duets. Frontiers in human neuroscience 7 (2013), 234.Google Scholar
- Erica Volta, Radoslaw Niewiadomski, Temitayo Olugbade, Carla Gilio, Elena Cocchi, Nadia Berthouze, Monica Gori, and Gualtiero Volpe. 2019. Analysis of cognitive states during bodily exploration of mathematical concepts in visually impaired children. In 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, 1–7.Google ScholarCross Ref
- Alan M Wing, Satoshi Endo, Adrian Bradbury, and Dirk Vorberg. 2014. Optimal feedback correction in string quartet synchronization. Journal of The Royal Society Interface 11, 93 (2014), 20131125.Google ScholarCross Ref
Index Terms
- Sensorimotor Synchronization in Blind Musicians: Does Lack of Vision Influencenon-verbal Musical Communication?
Recommendations
Empowering Blind Musicians to Compose and Notate Music with SoundCells
ASSETS '22: Proceedings of the 24th International ACM SIGACCESS Conference on Computers and AccessibilityCommercial technologies for notating music pose usage barriers to blind and visually impaired (BVI) musicians because they use graphic user interfaces and only produce visual, print scores. However, more research to date has studied how to make existing ...
Reflexive loopers for solo musical improvisation
CHI '13: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsLoop pedals are real-time samplers that playback audio played previously by a musician. Such pedals are routinely used for music practice or outdoor "busking". However, loop pedals always playback the same material, which can make performances ...
Synchronization and fluctuation of rhythm in musical cooperative performance
HCII'11: Proceedings of the 14th international conference on Human-computer interaction: design and development approaches - Volume Part IA live musical performance gives us better impression than recorded music heard from a portable music player. From player's point of view, live performance also gives better impression than playing music with metronome or recorded music. Thus, the ...
Comments