Abstract
The performance of non-invasive electroencephalogram-based (EEG) brain–computer interfaces (BCIs) has improved significantly in recent years. However, remaining challenges include the non-stationarity and the low signal-to-noise ratio of the EEG, which limit the bandwidth and hence the available applications. Optimization of both individual components of BCIs and the interrelationship between them is crucial to enhance bandwidth. In other words, neuroscientific knowledge and machine learning need to be optimized by considering concepts from human–computer interaction research and usability. In this paper, we present results of ongoing relevant research in our lab that addresses several important issues for BCIs based on the detection of transient changes in oscillatory EEG activity. First, we report on the long-term stability and robustness of detection of oscillatory EEG components modulated by distinct mental tasks, and show that the use of mental task pairs “mental subtraction versus motor imagery” achieves robust and reliable performance (Cohen’s κ > 0.6) in seven out of nine subjects over a period of 4 days. Second, we report on restricted Boltzmann machines (RBMs) as promising tools for the recognition of oscillatory EEG patterns. In an off-line BCI simulation we computed average peak accuracies, averaged over ten subjects, of 80.8 ± 7.2 %. Third, we present the basic framework of the context-aware hybrid Graz-BCI that allows interacting with the massive multiplayer online role playing game World of Warcraft. We show how a more integrated design approach that considers all components of BCIs, their interrelationships, other input signals and contextual information can increase interaction efficacy.
Similar content being viewed by others
References
Allison BZ, Brunner C, Kaiser V, Müller-Putz GR, Neuper C, Pfurtscheller G (2010) Toward a hybrid brain–computer interface based on imagined movement and visual attention. J Neural Eng 7:026,007. doi:10.1088/1741-2560/7/2/026007
Allison BZ, Leeb R, Brunner C, Müller-Putz GR, Bauernfeind G, Kelly JW, Neuper C (2012) Toward smarter BCIs: extending BCIs through hybridization and intelligent control. J Neural Eng 9(1):013001. doi:10.1088/1741-2560/9/1/013001
Balderas D, Zander T, Bachl F, Neuper C, Scherer R (2011) Restricted boltzmann machines as useful tool for detecting oscillatory eeg components. In: Proc. of the 5th international brain–computer interface conference, Graz, Austria, pp 68–71
Blankertz B, Dornhege G, Krauledat M, Müller KR, Curio G (2007) The non-invasive Berlin brain–computer interface: fast acquisition of effective performance in untrained subjects. NeuroImage 37:539–550. doi:10.1016/j.neuroimage.2007.01.051
Blankertz B, Tomioka R, Lemm S, Kawanabe M, Müller KR (2008) Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal Process Mag 25:41–56. doi:10.1109/MSP.2008.4408441
Carreira-Perpinan MA, Hinton GE (2005) On contrastive divergence learning. In: Proceedings of the tenth international workshop on artificial intelligence and statistics, pp 33–40
Chung M, Cheung W, Scherer R, Rao RPN (2011) A hierarchical architecture for adaptive brain–computer interfacing. In: Proceedings of the 22nd international joint conference on artificial intelligence, IJCAI 2011, Barcelona, Catalonia, Spain, July 16–22, 2011 pp 1647–1652
Cohen J (1960) A coefficient of agreement for nominal scales. Psychol Meas 20:37–46
Darvas F, Scherer R, Ojemann JG, Rao RP, Miller KJ, Sorensen LB (2009) High gamma mapping using EEG. NeuroImage 49:930–938. doi:10.1016/j.neuroimage.2009.08.041
Fatourechi M, Bashashati A, Ward RK, Birch GE (2007) EMG and EOG artifacts in brain computer interface systems: a survey. Clin Neurophysiol 118:480–494
Fazli S, Mehnert J, Steinbrink J, Curio G, Villringer A, Müller KR, Blankertz B (2012) Enhanced performance by a hybrid NIRSEEG brain computer interface. NeuroImage 59:519–529. doi:10.1016/j.neuroimage.2011.07.084
Friedrich EVC, Scherer R, Neuper C (2012) The effect of distinct mental strategies on classification performance for brain–computer interfaces. Int J Psychophysiol 84(1):86–94
Gevins A, Smith ME, McEvoy L, Yu D (1997) High-resolution eeg mapping of cortical activation related to working memory: effects of task difficulty, type of processing, and practice. Cereb Cortex 7(4):374–385
Grozea C, Voinescu CD, Fazli S (2011) Bristle-sensors—low-cost flexible passive dry EEG electrodes for neurofeedback and BCI applications. J Neural Eng 8:025,008. doi:10.1088/1741-2560/8/2/025008
Hinton GE (2002) Training products of experts by minimizing contrastive divergence. Neural Comput 14(8):1771–1800
Hinton GE (2010) A practical guide to training restricted boltzmann machines. http://www.cs.toronto.edu/hinton/absps/guideTR.pdf
Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18:1527–1554
Hjorth B (1975) An on-line transformation of EEG scalp potentials into orthogonal source derivations. Electroencephalogr Clin Neurophysiol 39:526–530. doi:10.1016/0013-4694(75)90056-5
Johnson RR, Popovic DP, Olmstead RE, Stikic M, Levendowski DJ, Berka C (2011) Drowsiness/alertness algorithm development and validation using synchronized eeg and cognitive performance to individualize a generalized model. Biol Psychol 87(2):241–250. doi:10.1016/j.biopsycho.2011.03.003
Krauledat M, Tangermann M, Blankertz B, Müller KR (2008) Towards zero training for brain–computer interfacing. PLoS ONE 3:e2967. doi:10.1371/journal.pone.0002967
Larochelle H, Bengio Y (2008) Classification using discriminative restricted boltzmann machines. In: ICML 08: proceedings of the 25th international conference on machine learning. ACM
Leeb R, Friedman D, Müller-Putz GR, Scherer R, Slater M, Pfurtscheller G (2007) Self-paced (asynchronous) BCI control of a wheelchair in virtual environments: a case study with a tetraplegics. Comput Intell Neurosci 2007:79,642. doi:10.1155/2007/79642
Leeb R, Sagha H, Chavarriaga R, Milln JDR (2011) A hybrid brain–computer interface based on the fusion of electroencephalographic and electromyographic activities. J Neural Eng 8(2):025011. doi:10.1088/1741-2560/8/2/025011
Lotte F, Congedo M, Lcuyer A, Lamarche F, Arnaldi B (2007) A review of classification algorithms for EEG-based brain–computer interfaces. J Neural Eng 4:R1–R13. doi:10.1088/1741-2560/4/2/R01
Mason SG, Bashashati A, Fatourechi M, Navarro KF, Birch GE (2007) A comprehensive survey of brain interface technology designs. Ann Biomed Eng 35:137–169. doi:10.1007/s10439-006-9170-0
Millán J, Mourino J (2003) Asynchronous BCI and local neural classifiers: an overview of the adaptive brain interface project. IEEE Trans Neural Syst Rehabil Eng 11:159–161
Müller-Gerking J, Pfurtscheller G, Flyvbjerg H (1999) Designing optimal spatial filters for single-trial EEG classification in a movement task. Clin Neurophysiol 110:787–798
Müller-Putz GR, Eder E, Wriessnegger SC, Pfurtscheller G (2008) Comparison of DFT and lock-in amplifier features and search for optimal electrode positions in SSVEP-based BCI. J Neurosci Methods 168:174–181. doi:10.1016/j.jneumeth.2007.09.024
Müller-Putz GR, Scherer R, Pfurtscheller G, Neuper C (2010) Temporal coding of brain patterns for direct limb control in humans. Front Neurosci 4. doi:10.3389/fnins.2010.00034
Navarro NA, Ceccaroni L, Velickovski F, Torrellas S, Miralles F, Allison BZ, Scherer R, Faller J (2011) Context-awareness as an enhancement of brain–computer interfaces. In: International workshop on ambient assisted living, Malaga, Spain. Lecture notes in computer science, pp 216–223
Navdeep J, Hinton GE (2011) Learning a better representation of speech sound waves using restricted boltzmann machines. In: Proceedings of the 12th international conference on artificial intelligence and statistics (AISTATS)
Neuper C, Scherer R, Reiner M, Pfurtscheller G (2005) Imagery of motor actions: differential effects of kinesthetic versus visual-motor mode of imagery on single-trial EEG. Brain Res Cognit Brain Res 25:668–677
Neuper C, Scherer R, Wriessnegger S, Pfurtscheller G (2009) Motor imagery and action observation: modulation of sensorimotor brain rhythms during mental control of a brain–computer interface. Clin Neurophysiol 120:239–247. doi:10.1016/j.clinph.2008.11.015
Pfurtscheller G, Lopes da Silva FH (1999) Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin Neurophysiol 110:1842–1857. doi:10.1016/S1388-2457(99)00141-8
Pfurtscheller G, Scherer R, Müller-Putz GR, Lopes da Silva FH (2008) Short-lived brain state after cued motor imagery in naive subjects. Eur J Neurosci 28:1419–1426. doi:10.1111/j.1460-9568.2008.06441.x
Pfurtscheller G, Allison BZ, Brunner C, Bauernfeind G, Solis-Escalante T, Scherer R, Zander TO, Müller-Putz G, Neuper C, Birbaumer N (2010) The hybrid BCI. Front Neurosci 4:30. doi:10.3389/fnpro.2010.00003
Popescu F, Fazli S, Badower Y, Blankertz B, Müller KR (2007) Single trial classification of motor imagination using 6 dry EEG electrodes. PLoS ONE 2:e637. doi:10.1371/journal.pone.0000637
Pregenzer M, Pfurtscheller G (1999) Frequency component selection for an EEG-based brain to computer interface. IEEE Trans Neural Syst Rehabil Eng 7:413–419
Pregenzer M, Pfurtscheller G, Flotzinger D (1996) Automated feature selection with a distinction sensitive learning vector quantizer. Neurocomputing 11:19–29. doi:10.1016/0925-2312(94)00071-9
Ramoser H, Müller-Gerking J, Pfurtscheller G (2000) Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Trans Rehabil Eng 8:441–446
Rao RPN, Scherer R (2010) Brain–computer interfacing. IEEE Signal Proc Mag 27(4):150–152
Scherer R, Müller GR, Neuper C, Graimann B, Pfurtscheller G (2004) An asynchronously controlled EEG-based virtual keyboard: improvement of the spelling rate. IEEE Trans Neural Syst Rehabil Eng 51:979–984
Scherer R, Schlögl A, Lee F, Bischof H, Jansa J, Pfurtscheller G (2007) The self-paced Graz brain–computer interface: methods and applications. Comput Intell Neurosci 2007:79,826
Scherer R, Lee F, Schlögl A, Leeb R, Bischof H, Pfurtscheller G (2008) Toward self-paced brain–computer communication: navigation through virtual worlds. IEEE Trans Biomed Eng 55:675–682. doi:10.1109/TBME.2007.903709
Scherer R, Pfurtscheller G, Neuper C (2008) Motor imagery induced changes in oscillatory ee components: speed vs. accuracy. In: Proceedings of the Graz brain–computer interface workshop 2008
Scherer R, Müller-Putz GR, Pfurtscheller G (2009) Flexibility and practicality: Graz brain–computer interface approach. Int Rev Neurobiol 86:119–131. doi:10.1016/S0074-7742(09)86009-1
Scherer R, Friedrich EVC, Allison BZ, Pröll M, Chung M, Cheung W, Rao RPN, Neuper C (2011) Non-invasive brain–computer interfaces: enhanced gaming and robotic control. Lect Notes Comput Sci 6691:362–369
Schlögl A, Kronegg J, Huggins JE, Mason SG (2007) Evaluation criteria for BCI research. In: Toward brain–computer interfacing. MIT Press, Cambridge
Shenoy P, Krauledat M, Blankertz B, Rao RP, Müller KR (2006) Towards adaptive classification for BCI. J Neural Eng 3:13–23
Trejo L, Kochavi R, Kubitz K, Montgomery L, Rosipal R, Matthews B (2005) EEG-based estimation of cognitive fatigue. In: SPIE conference proceedings
Usakli AB, Gurkan S, Aloise F, Vecchiato G, Babiloni F (2009) A hybrid platform based on EOG and EEG signals to restore communication for patients afflicted with progressive motor neuron diseases. In: Conference proceedings of the IEEE engineering in medicine and biology society
Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM (2002) Brain–computer interfaces for communication and control. Clin Neurophysiol 113:767–791. doi:10.1016/S1388-2457(02)00057-3
Zander TO, Jatzev S (2012) Context-aware brain–computer interfaces: exploring the information space of user, technical system and environment. J Neural Eng 9(1):016003. doi:10.1088/1741-2560/9/1/016003
Zander TO, Kothe C (2011) Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general. J Neural Eng 8:025,005. doi:10.1088/1741-2560/8/2/025005
Acknowledgments
This work was supported in part by the ICT Collaborative Project BrainAble (247447), the GaLA project (258169), the Wings for Life Spinal Cord Foundation, and ARO award W911NF-11-1-0307.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Scherer, R., Faller, J., Balderas, D. et al. Brain–computer interfacing: more than the sum of its parts. Soft Comput 17, 317–331 (2013). https://doi.org/10.1007/s00500-012-0895-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-012-0895-4