Abstract
In real life settings, human operators work in cooperation to optimize both safety and performance. The goal of this study is to assess teammates’ cooperation level using cerebral measures and machine learning techniques. We designed an experimental protocol with a modified version of the NASA MATB-II that was performed in 8 five-minute blocks. Each participant was either Pilot Flying (PF) or Pilot Monitoring (PM) with specific sub-tasks to attend to. In half the blocks they were instructed to cooperate by helping the other with one of his/her sub-tasks. Five teams of two healthy volunteers were recruited among the students of the ISAE-SUPAERO engineering school. In addition to behavioral data, their electroencephalogram (EEG) was recorded. The cooperation level of the participants was estimated using a brain-computer interface pipeline with a classification step applied on basic connectivity features, i.e. covariance matrices computed between participants’ EEG sensors. Behavioral results revealed a significant impact of cooperative instructions. Also, the implemented estimation pipeline allowed to estimate cooperative states using covariance matrices with an average accuracy of 66.6% using the signal filtered in the theta band, 64.5% for the alpha band and 65.3% for the low beta band. These preliminary estimation results are above the adjusted chance level and pave the way towards adaptive training tools based on hyperscanning for aeronautical settings.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Palumbo, R.V., et al.: Interpersonal autonomic physiology: a systematic review of the literature. Pers. Soc. Psychol. Rev. 21(2), 99–141 (2017)
Montague, P.R., et al.: Hyperscanning: simultaneous fMRI during linked social interactions. NeuroImage 16, 1159–1164 (2002)
Babiloni, F., Astolfi, L.: Social neuroscience and hyperscanning techniques: past, present and future. Neurosci. Biobehav. Rev. 44, 76–93 (2014)
Brouwer, A.M., Hogervorst, M.A., Van Erp, J.B., Heffelaar, T., Zimmerman, P.H., Oostenveld, R.: Estimating workload using EEG spectral power and ERPs in the n-back task. J. Neural Eng. 9(4), 045008 (2012)
Roy, R.N., Charbonnier, S., Bonnet, S.: Detection of mental fatigue using an active BCI inspired signal processing chain. IFAC Proc. Volumes 47(3), 2963–2968 (2014)
Roy, R.N., Charbonnier, S., Campagne, A., Bonnet, S.: Efficient mental workload estimation using task-independent EEG features. J. Neural Eng. 13(2), 026019 (2016)
Charbonnier, S., Roy, R.N., Bonnet, S., Campagne, A.: EEG index for control operators’ mental fatigue monitoring using interactions between brain regions. Expert Syst. Appl. 52, 91–98 (2016)
Dehais, F., Roy, R.N., Scannella, S.: Inattentional deafness to auditory alarms: inter-individual differences, electrophysiological signature and single trial classification. Behav. Brain Res. 360, 51–59 (2019)
Stevens, R.H., Galloway, T.L., Willemsen-Dunlap, A.: Neuroergonomics: quantitative modeling of individual, shared, and team neurodynamic information. Hum. Fact. 60(7), 1022–1034 (2018)
Stevens, R.H., Galloway, T., Berka, C., Behneman, A.: A neurophysiologic approach for studying team cognition. In: Interservice/Industry Training Simulation and Education Conference (I/ITSEC), paper no. 10135 (2010)
Fairclough, S.H.: Fundamentals of physiological computing. Interact. Comput. 21(1–2), 133–145 (2009)
Zander, T.O., Kothe, C., Jatzev, S., Gaertner, M.: Enhancing human-computer interaction with input from active and passive brain-computer interfaces. In: Tan, D., Nijholt, A. (eds.) Brain-Computer Interfaces, pp. 181–199. Springer, London (2010). https://doi.org/10.1007/978-1-84996-272-8_11
George, L., Lécuyer, A.: An overview of research on “passive” brain-computer interfaces for implicit human-computer interaction. In: International Conference on Applied Bionics and Biomechanics - Workshop W1 Brain-Computer Interfacing and Virtual Reality (2010)
Stephens, C., et al.: Biocybernetic adaptation strategies: machine awareness of human engagement for improved operational performance. In: Schmorrow, D.D., Fidopiastis, C.M. (eds.) AC 2018. LNCS (LNAI), vol. 10915, pp. 89–98. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91470-1_9
Stevens, R.H., Galloway, T., Berka, C., Sprang, M.: Can neurophysiologic synchronies provide a platform for adapting team performance? In: Schmorrow, D.D., Estabrooke, I.V., Grootjen, M. (eds.) FAC 2009. LNCS (LNAI), vol. 5638, pp. 658–667. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-02812-0_75
Verdiére, K.J., Dehais, F., Roy, R.N.: Spectral EEG-based classification for operator dyads’ workload and cooperation level estimation. In: IEEE, Systems, Man and Cybernetics Conference, pp. 3919–3924 (2019)
Toppi, J., et al.: Investigating cooperative behavior in ecological settings: an EEG hyperscanning study. In: PloS one 11(4), e0154236 (2016)
Barachant, A., Bonnet, S., Congedo, M., Jutten, C.: Multiclass brain-computer interface classification by Riemannian geometry. IEEE Trans. Biomed. Eng. 59(4), 920–928 (2011)
Santiago-Espada, Y., Myer, R.R., Latorella, K. A., Comstock Jr., J. R.: The multi-attribute task battery II (MATB-II) software for human performance and workload research: a user’s guide (2011)
NASA: MATB-II: Revised multi-attribute task battery (2019). https://matb.larc.nasa.gov/
Liu, T., Pelowski, M.: Clarifying the interaction types in two-person neuroscience research. Front. Hum. Neurosci. 8, 276 (2014)
Lotte, F., Roy, R.N.: Brain–computer interface contributions to neuroergonomics. In: Neuroergonomics, pp. 43–48. Academic Press (2019)
Mullen, T., et al.: Real-time modeling and 3D visualization of source dynamics and connectivity using wearable EEG. In: 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 2184–2187 (2013)
Barachant, A.: Covariance toolbox (2013). https://github.com/alexandrebarachant/covariancetoolbox
Combrisson, E., Jerbi, K.: Exceeding chance level by chance: the caveat of theoretical chance levels in brain signal classification and statistical assessment of decoding accuracy. J. Neurosci. Methods 250, 126–136 (2015)
Gerjets, P., Walter, C., Rosenstiel, W., Bogdan, M., Zander, T.O.: Cognitive state monitoring and the design of adaptive instruction in digital environments: lessons learned from cognitive workload assessment using a passive brain-computer interface approach. Front. Neurosci. 8, 385 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Roy, R.N., Verdière, K.J., Dehais, F. (2020). EEG Covariance-Based Estimation of Cooperative States in Teammates. In: Sottilare, R.A., Schwarz, J. (eds) Adaptive Instructional Systems. HCII 2020. Lecture Notes in Computer Science(), vol 12214. Springer, Cham. https://doi.org/10.1007/978-3-030-50788-6_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-50788-6_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-50787-9
Online ISBN: 978-3-030-50788-6
eBook Packages: Computer ScienceComputer Science (R0)