skip to main content
10.1145/3410700.3425434acmconferencesArticle/Chapter ViewAbstractPublication Pagessiggraph-asiaConference Proceedingsconference-collections
research-article

NeuralDrum: Perceiving Brain Synchronicity in XR Drumming

Published: 17 November 2020 Publication History

Abstract

Brain synchronicity is a neurological phenomena where two or more individuals have their brain activation in phase when performing a shared activity. We present NeuralDrum, an extended reality (XR) drumming experience that allows two players to drum together while their brain signals are simultaneously measured. We calculate the Phase Locking Value (PLV) to determine their brain synchronicity and use this to directly affect their visual and auditory experience in the game, creating a closed feedback loop. In a pilot study, we logged and analysed the users’ brain signals as well as had them answer a subjective questionnaire regarding their perception of synchronicity with their partner and the overall experience. From the results, we discuss design implications to further improve NeuralDrum and propose methods to integrate brain synchronicity into interactive experiences.

Supplementary Material

MP4 File (3410700.3425434.mp4)
Presentation video

References

[1]
Laura Astolfi, Jlenia Toppi, Gianluca Borghini, Giovanni Vecchiato, R Isabella, F De Vico Fallani, Febo Cincotti, Serenella Salinari, Donatella Mattia, B He, 2011. Study of the functional hyperconnectivity between couples of pilots during flight simulation: An EEG hyperscanning study. In 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, 2338–2341.
[2]
Fabio Babiloni, Febo Cincotti, Donatella Mattia, F De Vico Fallani, Andrea Tocci, Luigi Bianchi, S Salinari, MG Marciani, A Colosimo, and Laura Astolfi. 2007. High resolution EEG hyperscanning during a card game. In 2007 29th annual international conference of the ieee engineering in medicine and biology society. IEEE, 4957–4960.
[3]
Hao Chen, Arindam Dey, Mark Billinghurst, and Robert W Lindeman. 2017. Exploring the design space for multi-sensory heart rate feedback in immersive virtual reality. In Proceedings of the 29th Australian Conference on Computer-Human Interaction. ACM, 108–116.
[4]
Thomas D Duane and Thomas Behrendt. 1965. Extrasensory electroencephalographic induction between identical twins.Science (1965).
[5]
Ryo Hajika, Kunal Gupta, Prasanth Sasikumar, and Yun Suen Pai. 2019. HyperDrum: Interactive Synchronous Drumming in Virtual Reality using Everyday Objects. In SIGGRAPH Asia 2019 XR. ACM, 15–16.
[6]
Xiao Jiang, Gui-Bin Bian, and Zean Tian. 2019. Removal of artifacts from EEG signals: a review. Sensors 19, 5 (2019), 987.
[7]
Seungwon Kim, Gun Lee, Nobuchika Sakata, and Mark Billinghurst. 2014. Improving co-presence with augmented visual communication cues for sharing experience through video conference. In 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 83–92.
[8]
Sonja C Kleih and Andrea Kübler. 2015. Psychological factors influencing brain-computer interface (BCI) performance. In 2015 IEEE International Conference on Systems, Man, and Cybernetics. IEEE, 3192–3196.
[9]
P Read Montague, Gregory S Berns, Jonathan D Cohen, Samuel M McClure, Giuseppe Pagnoni, Mukesh Dhamala, Michael C Wiest, Igor Karpov, Richard D King, Nathan Apple, 2002. Hyperscanning: simultaneous fMRI during linked social interactions.
[10]
Robert Oostenveld and Peter Praamstra. 2001. The five percent electrode system for high-resolution EEG and ERP measurements. Clinical neurophysiology 112, 4 (2001), 713–719.
[11]
Misha Sra, Ken Perlin, Luiz Velho, and Mark Bolas. 2018. Novel interaction techniques for collaboration in VR. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, W33.
[12]
Jlenia Toppi, Gianluca Borghini, Manuela Petti, Eric J He, Vittorio De Giusti, Bin He, Laura Astolfi, and Fabio Babiloni. 2016. Investigating cooperative behavior in ecological settings: an EEG hyperscanning study. PloS one 11, 4 (2016), e0154236.
[13]
Peng Wang, Shusheng Zhang, Xiaoliang Bai, Mark Billinghurst, Li Zhang, Shuxia Wang, Dechuan Han, Hao Lv, and Yuxiang Yan. 2019. A gesture-and head-based multimodal interaction platform for MR remote collaboration. The International Journal of Advanced Manufacturing Technology 105, 7-8(2019), 3031–3043.
[14]
Jun Xie, Guanghua Xu, Jing Wang, Min Li, Chengcheng Han, and Yaguang Jia. 2016. Effects of mental load and fatigue on steady-state evoked potential based brain computer interface tasks: a comparison of periodic flickering and motion-reversal based visual attention. PloS one 11, 9 (2016), e0163426.
[15]
Kyongsik Yun, Katsumi Watanabe, and Shinsuke Shimojo. 2012. Interpersonal body and neural synchronization as a marker of implicit social interaction. Scientific reports 2(2012), 959.

Cited By

View all
  • (2024)BCHJam: a Brain-Computer Music Interface for Live Music Performance in Shared Mixed Reality Environments2024 IEEE 5th International Symposium on the Internet of Sounds (IS2)10.1109/IS262782.2024.10704087(1-9)Online publication date: 30-Sep-2024
  • (2024)The virtual drum circle: polyrhythmic music interactions in mixed realityJournal of New Music Research10.1080/09298215.2024.233924452:4(316-336)Online publication date: 15-Apr-2024
  • (2023)VR Drumming Pedagogy: Action Observation, Virtual Co-Embodiment, and Development of Drumming “Halvatar”Electronics10.3390/electronics1217370812:17(3708)Online publication date: 1-Sep-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SA '20: SIGGRAPH Asia 2020 Technical Communications
December 2020
56 pages
ISBN:9781450380805
DOI:10.1145/3410700
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 November 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Virtual Reality
  2. brain synchronicity
  3. hyperscanning

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

SA '20
Sponsor:
SA '20: SIGGRAPH Asia 2020
December 1 - 9, 2020
Virtual Event, Republic of Korea

Acceptance Rates

Overall Acceptance Rate 178 of 869 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)65
  • Downloads (Last 6 weeks)6
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)BCHJam: a Brain-Computer Music Interface for Live Music Performance in Shared Mixed Reality Environments2024 IEEE 5th International Symposium on the Internet of Sounds (IS2)10.1109/IS262782.2024.10704087(1-9)Online publication date: 30-Sep-2024
  • (2024)The virtual drum circle: polyrhythmic music interactions in mixed realityJournal of New Music Research10.1080/09298215.2024.233924452:4(316-336)Online publication date: 15-Apr-2024
  • (2023)VR Drumming Pedagogy: Action Observation, Virtual Co-Embodiment, and Development of Drumming “Halvatar”Electronics10.3390/electronics1217370812:17(3708)Online publication date: 1-Sep-2023
  • (2023)How Field of View Affects Awareness of an Avatar During a Musical Task in Augmented Reality2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW58643.2023.00159(633-634)Online publication date: Mar-2023
  • (2023)Investigating the Effects of Limited Field of View on Jamming Experience in Extended Reality2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct60411.2023.00107(493-495)Online publication date: 16-Oct-2023
  • (2021)Lifelogging meets alternate and cross-realities: an investigation into broadcasting personal visual realities to remote audiencesMultimedia Tools and Applications10.1007/s11042-021-11310-383:15(46707-46730)Online publication date: 14-Aug-2021

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media