Abstract
Introduction: Brain-computer interfaces (BCIs) provide a broad range of applications for human-computer interactions. Exploring cognitive control and underlying neurophysiological mechanisms brings essential contributions to this research field. In this paper, neurophysiological findings connected to cognitive control processes using the Stroop experiment were investigated. Electroencephalography (EEG) and functional infrared spectroscopy (fNIRS) were employed for measuring brain activities. The Stroop-test was classified against resting-state activities.
Materials and Methods: The wireless g.Nautilus fNIRS system (g.tec medical engineering GmbH) with 16 channels of EEG, combined with 8 channels of fNIRS, was used for data acquisition. Six healthy subjects participated in the conducted Stroop experiment.
Results: A considerable hemodynamic response was present during the Stroop-test, as seen in the offline analysis. The EEG-based classification delay was considerably lower than those with oxygenated and deoxygenated hemoglobin-based classifiers. The online experiment analysis showed that the accuracy rose clearly within the first 2 s of the task. On average, a maximum accuracy of 81.0% was achieved at 6.2 s after the task onset.
Discussion: and Conclusion: In general, the hybrid approach seems superior by facilitating information from all three modalities. In conclusion, the capability of successfully determining frontal lobe activity is a promising indication to use hybrid BCIs for further research applications.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Wolpaw, J.R., Birbaumer, N., McFarland, D.J., et al.: Brain–computer interfaces for communication and control. Clin. Neurophysiol. 113, 767–791 (2002). https://doi.org/10.1016/S1388-2457(02)00057-3
Wolpaw, J.R., Loeb, G.E., Allison, B.Z., et al.: BCI meeting 2005–workshop on signals and recording methods. IEEE Trans. Neural Syst. Rehabil. Eng. 14, 138–141 (2006). https://doi.org/10.1109/TNSRE.2006.875583
Leuthardt, E.C., Schalk, G., Roland, J., et al.: Evolution of brain-computer interfaces: going beyond classic motor physiology. Neurosurg. Focus 27, E4 (2009). https://doi.org/10.3171/2009.4.FOCUS0979
Lebedev, M.A., Nicolelis, M.A.L.: Brain–machine interfaces: past, present and future. Trends Neurosci. 29, 536–546 (2006). https://doi.org/10.1016/j.tins.2006.07.004
Bandara, D.S.V., Kiguchi, K.: Brain signal acquisition methods in BCIs to estimate human motion intention – a survey. In: 2018 International Symposium on Micro-NanoMechatronics and Human Science (MHS), pp. 1–7. IEEE, Nagoya (2018). https://doi.org/10.1109/MHS.2018.8887072
Hong, K.-S., Naseer, N., Kim, Y.-H.: Classification of prefrontal and motor cortex signals for three-class fNIRS–BCI. Neurosci. Lett. 6 (2015). https://doi.org/10.1016/j.neulet.2014.12.029
Chiarelli, A.M., Croce, P., Merla, A., Zappasodi, F.: Deep learning for hybrid EEG-fNIRS brain-computer interface: application to motor imagery classification. J. Neural. Eng. 15, 036028 (2018). https://doi.org/10.1088/1741-2552/aaaf82
Verma, P., Heilinger, A., Reitner, P., et al.: Performance investigation of brain-computer interfaces that combine EEG and fNIRS for motor imagery tasks. In: 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), pp. 259–263. IEEE, Bari (2019). https://doi.org/10.1109/SMC.2019.8914083
Midha, S., Maior, H.A., Wilson, M.L., Sharples, S.: Measuring mental workload variations in office work tasks using fNIRS. Int. J. Hum. Comput. Stud. 147, 102580 (2021). https://doi.org/10.1016/j.ijhcs.2020.102580
Cicalese, P.A., Li, R., Ahmadi, M.B., et al.: An EEG-fNIRS hybridization technique in the four-class classification of alzheimer’s disease. J. Neurosci. Methods 336, 108618 (2020). https://doi.org/10.1016/j.jneumeth.2020.108618
Farina, F.R., Emek-Savaş, D.D., Rueda-Delgado, L., et al.: A comparison of resting state EEG and structural MRI for classifying Alzheimer’s disease and mild cognitive impairment. Neuroimage 215, 116795 (2020). https://doi.org/10.1016/j.neuroimage.2020.116795
Todri, J., Lena, O., Martínez Gil, J.L.: A single blind randomized controlled trial of global postural re-education: cognitive effects on Alzheimer disease patients. Eur. J. Psychiatry 33, 83–90 (2019). https://doi.org/10.1016/j.ejpsy.2019.01.001
Heidlmayr, K., Kihlstedt, M., Isel, F.: A review on the electroencephalography markers of stroop executive control processes. Brain Cogn. 146, 105637 (2020). https://doi.org/10.1016/j.bandc.2020.105637
Tafuro, A., Ambrosini, E., Puccioni, O., Vallesi, A.: Brain oscillations in cognitive control: a cross-sectional study with a spatial stroop task. Neuropsychologia 133, 107190 (2019). https://doi.org/10.1016/j.neuropsychologia.2019.107190
Pijnenborg, G.H.M., Larabi, D.I., Xu, P., et al.: Brain areas associated with clinical and cognitive insight in psychotic disorders: a systematic review and meta-analysis. Neurosci. Biobehav. Rev. 116, 301–336 (2020). https://doi.org/10.1016/j.neubiorev.2020.06.022
Yeung, M.K., Lee, T.L., Chan, A.S.: Neurocognitive development of flanker and Stroop interference control: a near-infrared spectroscopy study. Brain Cogn. 143, 105585 (2020). https://doi.org/10.1016/j.bandc.2020.105585
De, A., Konar, A., Samanta, A., et al.: An fNIRs study to classify stages of learning from visual stimuli using prefrontal hemodynamics. In: 2017 Third International Conference on Biosignals, Images and Instrumentation (ICBSII), pp. 1–7. IEEE, Chennai (2017). https://doi.org/10.1109/ICBSII.2017.8082272
Hirsch, G., Dirodi, M., Xu, R., Reitner, P., Guger, C.: Online classification of motor imagery using EEG and fNIRS: a hybrid approach with real time human-computer interaction. In: Stephanidis, C., Antona, M. (eds.) HCII 2020. CCIS, vol. 1224, pp. 231–238. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-50726-8_30
Blankertz, B., Tomioka, R., Lemm, S., et al.: Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal Process. Mag. 25, 41–56 (2008). https://doi.org/10.1109/MSP.2008.4408441
Geraedts, V.J., Marinus, J., Gouw, A.A., et al.: Quantitative EEG reflects non-dopaminergic disease severity in Parkinson’s disease. Clin. Neurophysiol. 8 (2018). https://doi.org/10.1016/j.clinph.2018.04.752
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)
Müller-Putz, G., Scherer, R., Brunner, C., et al.: Better than random: a closer look on BCI results. Int. J. Bioelectromagn. 10, 52–55 (2008)
Schroeter, M.L., Zysset, S., Kupka, T., et al.: Near-infrared spectroscopy can detect brain activity during a color-word matching stroop task in an event-related design. Hum. Brain Mapp. 17, 61–71 (2002). https://doi.org/10.1002/hbm.10052
Aghajani, H., Garbey, M., Omurtag, A.: Measuring mental workload with EEG+fNIRS. Front Hum. Neurosci. 11 (2017). https://doi.org/10.3389/fnhum.2017.00359
Al-Shargie, F., Kiguchi, M., Badruddin, N., et al.: Mental stress assessment using simultaneous measurement of EEG and fNIRS. Biomed. Opt. Express 7, 3882 (2016). https://doi.org/10.1364/BOE.7.003882
Acknowledgments
This work was partially funded via the European Commission project RHUMBO – H2020-MSCA-ITN-2018-813234.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Schreiner, L., Hirsch, G., Xu, R., Reitner, P., Pretl, H., Guger, C. (2021). Online Classification of Cognitive Control Processes Using EEG and fNIRS: A Stroop Experiment. In: Kurosu, M. (eds) Human-Computer Interaction. Theory, Methods and Tools. HCII 2021. Lecture Notes in Computer Science(), vol 12762. Springer, Cham. https://doi.org/10.1007/978-3-030-78462-1_45
Download citation
DOI: https://doi.org/10.1007/978-3-030-78462-1_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-78461-4
Online ISBN: 978-3-030-78462-1
eBook Packages: Computer ScienceComputer Science (R0)