Caterina Ceccato
ABSTRACT
A dual brain-computer interface (BCI) was developed to translate emotions and synchrony between two users into music. Using EEG signals of two individuals, the system generates live music note-by-note and controls musical parameters, such as pitch, intensity and interval. The users’ mean EEG amplitude determines the notes, and their emotional valence modulates the intensity (i.e. volume of music). Additionally, inter-brain synchrony is used to manipulate the interval between notes, with higher synchrony producing more pleasant music and lower synchrony producing less pleasant music. Further research is needed to test the system in an experimental setting, however, literature suggests that neurofeedback based on inter-brain synchrony and emotional valence could be used to promote positive aspects of group dynamics and mutual emotional understanding.
Supplemental Material
Available for Download
- Samuel Aaron and Alan F Blackwell. 2013. From sonic Pi to overtone: creative musical experiences with domain-specific and functional languages. In Proceedings of the first ACM SIGPLAN workshop on Functional art, music, modeling & design. 35–46.Google Scholar
Digital Library
- Michaël AS Acquadro, Marco Congedo, and Dirk De Riddeer. 2016. Music performance as an experimental approach to hyperscanning studies. Frontiers in human neuroscience 10 (2016), 242.Google Scholar
- Thomas G Arizmendi. 2011. Linking mechanisms: Emotional contagion, empathy, and imagery.Psychoanalytic Psychology 28, 3 (2011), 405.Google Scholar
- Hussain-Abdulah Arjmand, Jesper Hohagen, Bryan Paton, and Nikki S Rickard. 2017. Emotional Responses to Music: Shifts in Frontal Brain Asymmetry Mark Periods of Musical Change. Frontiers in Psychology 8 (2017).Google Scholar
- Caterina Ceccato, Ethel Pruss, Anita Vrins, and Jos Prinsen. 2023. BrainiBeats demo video. https://www.youtube.com/watch?v=Oh-JXN6_UIEGoogle Scholar
- Thibault Chabin, Damien Gabriel, Alexandre Comte, Emmanuel Haffen, Thierry Moulin, and Lionel Pazart. 2022. Interbrain emotional connection during music performances is driven by physical proximity and individual traits. Annals of the New York Academy of Sciences 1508, 1 (2022), 178–195.Google ScholarCross Ref
- Rubén Hinojosa Chapel. 2003. Realtime Algorithmic Music Systems From Fractals and Chaotic Functions: Toward an Active Musical Instrument.Google Scholar
- Laura K. Cirelli. 2018. How interpersonal synchrony facilitates early prosocial behavior.Current opinion in psychology 20 (2018), 35–39.Google Scholar
- James A Coan and John JB Allen. 2004. Frontal EEG asymmetry as a moderator and mediator of emotion. Biological psychology 67, 1-2 (2004), 7–50.Google Scholar
- William A Cunningham, Stacey D Espinet, Colin G DeYoung, and Philip David Zelazo. 2005. Attitudes to the right-and left: frontal ERP asymmetries associated with stimulus valence and processing goals. NeuroImage 28, 4 (2005), 827–834.Google ScholarCross Ref
- Artur Czeszumski, Sara Eustergerling, Anne Lang, David Menrath, Michael Gerstenberger, Susanne Schuberth, Felix Schreiber, Zadkiel Zuluaga Rendon, and Peter König. 2020. Hyperscanning: A Valid Method to Study Neural Inter-brain Underpinnings of Social Interaction. Frontiers in Human Neuroscience 14 (2020).Google Scholar
- Ian Daly, Duncan A. H. Williams, Alexis Kirke, James Weaver, Asad Malik, Faustina Hwang, Eduardo Reck Miranda, and Slawomir Jaroslaw Nasuto. 2016. Affective brain–computer music interfacing. Journal of Neural Engineering 13 (2016).Google Scholar
- Alexander P. Demos, Roger Chaffin, Kristen T. Begosh, Jennifer R. Daniels, and Kerry L. Marsh. 2012. Rocking to the beat: effects of music and partner’s movements on spontaneous interpersonal coordination.Journal of experimental psychology. General 141 1 (2012), 49–53.Google Scholar
- Guillaume Dumas, Jacqueline Nadel, Robert Soussignan, Jacques Martinerie, and Line Garnero. 2010. Inter-brain synchronization during social interaction. PloS one 5, 8 (2010), e12166.Google ScholarCross Ref
- Joel Eaton, Duncan A. H. Williams, and Eduardo Reck Miranda. 2015. The Space Between Us: Evaluating a multi-user affective brain-computer music interface.Google Scholar
- Hauke Egermann and Stephen McAdams. 2012. Empathy and emotional contagion as a link between recognized and felt emotions in music listening. Music Perception: An Interdisciplinary Journal 31, 2 (2012), 139–156.Google ScholarCross Ref
- Stefan K. Ehrlich, Kat Rose Agres, Cuntai Guan, and Gordon Cheng. 2019. A closed-loop, music-based brain-computer interface for emotion mediation. PLoS ONE 14 (2019).Google Scholar
- Walter J Freeman, Mark D Holmes, G Alexander West, and Sampsa Vanhatalo. 2006. Fine spatiotemporal structure of phase in human intracranial EEG. Clinical Neurophysiology 117, 6 (2006), 1228–1243.Google ScholarCross Ref
- Martin Gardner. 1978. Mathematical games: White and brown music, fractal curves and one-over-f fluctuations. Scientific american 238, 4 (1978), 16–32.Google Scholar
- Ihshan Gumilar, Ekansh Sareen, Reed Bell, Augustus Stone, Ashkan Hayati, Jingwen Mao, Amit Barde, Anubha Gupta, Arindam Dey, Gun Lee, 2021. A comparative study on inter-brain synchrony in real and virtual environments using hyperscanning. Computers & Graphics 94 (2021), 62–75.Google ScholarCross Ref
- Yasir Hafeez, Syed Saad Azhar Ali, Rumaisa Abu Hasan, Syed Hasan Adil, Muhammad Moinuddin, Mansoor Ebrahim, Muhamad Saiful Bahri Yusoff, Hafeezullah Amin, and Ubaid Al-Saggaf. 2021. Development of Enhanced Stimulus Content to Improve the Treatment Efficacy of EEG–Based Frontal Alpha Asymmetry Neurofeedback for Stress Mitigation. IEEE Access 9 (2021), 130638–130648.Google ScholarCross Ref
- Eddie Harmon-Jones. 2003. Early Career Award. Clarifying the emotive functions of asymmetrical frontal cortical activity.Psychophysiology 40 6 (2003), 838–48.Google Scholar
- Thilo Hinterberger and Gerold Baier. 2005. Parametric orchestral sonification of EEG in real time. IEEE MultiMedia 12, 2 (2005), 70–79.Google ScholarDigital Library
- Yinying Hu, Min Zhu, Yang Liu, Zixuan Wang, Xiaojun Cheng, Yafeng Pan, and Y. Hu. 2022. Musical Meter Induces Interbrain Synchronization during Interpersonal Coordination. eNeuro 9 (2022).Google Scholar
- Eva Istók, Elvira Brattico, Thomas Jacobsen, Kaisu Krohn, Mira Müller, and Mari Tervaniemi. 2009. Aesthetic responses to music: A questionnaire study. Musicae Scientiae 13, 2 (2009), 183–206.Google ScholarCross Ref
- Peter E. Keller, Giacomo Novembre, and Michael J. Hove. 2014. Rhythm in joint action: psychological and neurophysiological mechanisms for real-time interpersonal coordination. Philosophical Transactions of the Royal Society B: Biological Sciences 369 (2014).Google Scholar
- Sivan Kinreich, Amir Djalovski, Lior Kraus, Yoram Louzoun, and Ruth Feldman. 2017. Brain-to-brain synchrony during naturalistic social interactions. Scientific reports 7, 1 (2017), 17060.Google Scholar
- Ivana Konvalinka, Markus Bauer, Carsten Stahlhut, Lars Kai Hansen, Andreas Roepstorff, and Chris D. Frith. 2014. Frontal alpha oscillations distinguish leaders from followers: Multivariate decoding of mutually interacting brains. NeuroImage 94 (2014), 79–88.Google ScholarCross Ref
- Claudia Krogmeier, Brandon S Coventry, and Christos Mousas. 2022. Affective Image Sequence Viewing in Virtual Reality Theater Environment: Frontal Alpha Asymmetry Responses From Mobile EEG. Frontiers in Virtual Reality (2022).Google Scholar
- Jean-Philippe Lachaux, Eugenio Rodriguez, Jacques Martinerie, and Francisco J. Varela. 1999. Measuring phase synchrony in brain signals. Human Brain Mapping 8 (1999).Google Scholar
- Sylvain Le Groux, Jonatas Manzolli, Paul FMJ Verschure, Marti Sanchez, Andre Luvizotto, Anna Mura, Aleksander Valjamae, Christoph Guger, Robert Prueckl, and Ulysses Bernardet. 2010. Disembodied and Collaborative Musical Interaction in the Multimodal Brain Orchestra.. In NIME. Citeseer, 309–314.Google Scholar
- Chih Yi Lin and Stone Cheng. 2012. Multi-theme analysis of music emotion similarity for jukebox application. 2012 International Conference on Audio, Language and Image Processing (2012), 241–246.Google ScholarCross Ref
- Ulman Lindenberger, Shu-Chen Li, Walter Gruber, and Viktor Müller. 2009. Brains swinging in concert: cortical phase synchronization while playing guitar. BMC neuroscience 10, 1 (2009), 1–12.Google Scholar
- Bill Manaris, Juan Romero, Penousal Machado, Dwight Krehbiel, Timothy Hirzel, Walter Pharr, and Robert B Davis. 2005. Zipf’s law, music classification, and aesthetics. Computer Music Journal 29, 1 (2005), 55–69.Google ScholarDigital Library
- Danielle Mathersul, Leanne M Williams, Patrick J Hopkinson, and Andrew H Kemp. 2008. Investigating models of affect: relationships among EEG alpha asymmetry, depression, and anxiety.Emotion 8, 4 (2008), 560.Google Scholar
- Rocco Mennella, Elisabetta Patron, and Daniela Palomba. 2017. Frontal alpha asymmetry neurofeedback for the reduction of negative affect and anxiety. Behaviour research and therapy 92 (2017), 32–40.Google Scholar
- Eduardo Reck Miranda and Julien Castet. 2014. Guide to Brain-Computer Music Interfacing. In Springer London.Google Scholar
- Evan Morgan, Hatice Gunes, and Nick Bryan-Kinns. 2015. Using affective and behavioural sensors to explore aspects of collaborative music making. International Journal of Human-Computer Studies 82 (2015), 31–47.Google ScholarDigital Library
- Tim R. Mullen, Alexander Khalil, Tomas E. Ward, John Rehner Iversen, Grace Leslie, Richard Warp, Matt Whitman, Victor Minces, Aaron McCoy, Alejandro Ojeda, Nima Bigdely-Shamlo, Mike Chi, and David Rosenboom. 2015. MindMusic: Playful and Social Installations at the Interface Between Music and the Brain.Google Scholar
- Viktor Müller, Johanna C. Sänger, and Ulman Lindenberger. 2018. Hyperbrain network properties of guitarists playing in quartet. Annals of the New York Academy of Sciences 1423 (2018).Google Scholar
- Anton Nijholt. 2015. Competing and collaborating brains: multi-brain computer interfacing. In Brain-Computer Interfaces. Springer, 313–335.Google Scholar
- Naoyuki Osaka, Takehiro Minamoto, Ken Yaoi, Miyuki Azuma, Yohko Minamoto Shimada, and Mariko Osaka. 2015. How two brains make one synchronized mind in the inferior frontal cortex: fNIRS-based hyperscanning during cooperative singing. Frontiers in psychology 6 (2015), 1811.Google Scholar
- Andreas Pinegger, Hannah Hiebel, Selina C Wriessnegger, and Gernot R Müller-Putz. 2017. Composing only by thought: Novel application of the P300 brain-computer interface. PloS one 12, 9 (2017), e0181584.Google ScholarCross Ref
- Tal-Chen Rabinowitch, Ian Cross, and Pamela Burnard. 2013. Long-term musical group interaction has a positive influence on empathy in children. Psychology of Music 41 (2013), 484 – 498.Google ScholarCross Ref
- Mauricio A Ramírez-Moreno, Jesús G Cruz-Garza, Akanksha Acharya, Girija Chatufale, Woody Witt, Dan Gelok, Guillermo Reza, and José L Contreras-Vidal. 2022. Brain-to-brain communication during musical improvisation: a performance case study. F1000Research 11, 989 (2022), 989.Google Scholar
- Johanna Sänger, Viktor Müller, and Ulman Lindenberger. 2012. Intra-and interbrain synchronization and network properties when playing guitar in duets. Frontiers in human neuroscience (2012), 312.Google Scholar
- E. Glenn Schellenberg, Kathleen A. Corrigall, Sebastian P Dys, and Tina Malti. 2015. Group Music Training and Children’s Prosocial Skills. PLoS ONE 10 (2015).Google Scholar
- Benjamin T. Schmidt, Avniel Singh Ghuman, and Theodore J. Huppert. 2014. Whole brain functional connectivity using phase locking measures of resting state magnetoencephalography. Frontiers in Neuroscience 8 (2014).Google Scholar
- Davide Valeriani and Ana Matran-Fernandez. 2018. Past and future of multi-mind brain–computer interfaces. Brain–Computer Interfaces Handbook (2018), 685–700.Google Scholar
- Satvik Venkatesh, Eduardo Reck Miranda, and Edward Braund. 2022. SSVEP-based brain–computer interface for music using a low-density EEG system. Assistive Technology (2022), 1–11.Google Scholar
- Jinn-Rong Wang and Shulan Hsieh. 2013. Neurofeedback training improves attention and working memory performance. Clinical Neurophysiology 124 (2013), 2406–2420.Google ScholarCross Ref
- Valtteri Wikström, Katri Saarikivi, Mari Falcon, Tommi Makkonen, Silja Martikainen, Vesa Putkinen, Benjamin Ultan Cowley, and Mari Tervaniemi. 2022. Inter-brain synchronization occurs without physical co-presence during cooperative online gaming. Neuropsychologia 174 (2022).Google Scholar
- Duncan A. H. Williams. 2019. Evaluating BCI for Musical Expression: Historical Approaches, Challenges and Benefits. In Brain Art.Google Scholar
- Dan Wu, Chaoyi Li, Jie Liu, Jing Lu, and Dezhong Yao. 2014. Scale-free brain ensemble modulated by phase synchronization. Journal of Zhejiang University SCIENCE C 15 (2014), 821–831.Google ScholarCross Ref
- Kyongsik Yun, Katsumi Watanabe, and Shinsuke Shimojo. 2012. Interpersonal body and neural synchronization as a marker of implicit social interaction. Scientific reports 2, 1 (2012), 1–8.Google Scholar
- Wei Zhou, Chenyang Qiu, and Guangyuan Liu. 2021. Efficient regulation of emotion by positive music based on EEG valence-arousal model. 2021 3rd International Conference on Image, Video and Signal Processing (2021).Google ScholarDigital Library
Index Terms
- BrainiBeats: A dual brain-computer interface for musical composition using inter-brain synchrony and emotional valence
Recommendations
Visual context effects on the perception of musical emotional expressions
BioID_MultiComm'09: Proceedings of the 2009 joint COST 2101 and 2102 international conference on Biometric ID management and multimodal communicationIs there any evidence that context plays a role in the perception of the emotional feeling aroused by emotional musical expressions?. This work tries to answer the above question through a series of experiments where subjects were asked to label as ...
Human emotion recognition and analysis in response to audio music using brain signals
Human emotion recognition using brain signals is an active research topic in the field of affective computing. Music is considered as a powerful tool for arousing emotions in human beings. This study recognized happy, sad, love and anger emotions in ...
Emotion control system for MIDI excerpts: MOR2ART
Fun and Games '10: Proceedings of the 3rd International Conference on Fun and GamesEmotional expression when performing music (singing or playing musical instruments) requires skill, but such a skill is generally difficult to learn. Computer systems that can make it easy for non-musicians to express any emotion have been proposed[1]. ...
Comments