Abstract
The devices that can read Electroencephalography (EEG) signals have been widely used for Brain-Computer Interfaces (BCIs). Popularity in the field of BCIs has increased in recent years with the development of several consumer-grade EEG devices that can detect human cognitive states in real-time and deliver feedback to enhance human performance. Several previous studies have been conducted to understand the fundamentals and essential aspects of EEG in BCIs. However, the significant issue of how consumer-grade EEG devices can be used to control mechatronic systems effectively has been given less attention. In this article, we have designed and implemented an EEG BCI system using the OpenBCI Cyton headset and a user interface running a game to explore the concept of streamlining the interaction between humans and mechatronic systems with a BCI EEG-mechatronic system interface. Big Multimodal Social Data (BMSD) analytics can be applied to the high-frequency and high-volume EEG data, allowing us to explore aspects of data acquisition, data processing, and data validation and evaluate the Quality of Experience (QoE) of our system. We employ real-world participants to play a game to gather training data that was later put into multiple machine learning models, including a linear discriminant analysis (LDA), k-nearest neighbours (KNN), and a convolutional neural network (CNN). After training the machine learning models, a validation phase of the experiment took place where participants tried to play the same game but without direct control, utilising the outputs of the machine learning models to determine how the game moved. We find that a CNN trained to the specific user was able to control the game and performed with the highest activation accuracy from the machine learning models tested, along with the highest user-rated QoE, which gives us significant insight for future implementation with a mechatronic system.
- [1] . 2015. Brain computer interfacing: Applications and challenges. Egypt. Inform. J. 16, 2 (2015), 213–230.Google ScholarCross Ref
- [2] Swati Aggarwal and Nupur Chugh. 2022. Review of machine learning techniques for EEG based brain computer interface. Archives of Computational Methods in Engineering 29, 5 (2022), 3001–3020.Google Scholar
- [3] . 2009. An on-line BCI system for hand movement control using real-time recurrent probabilistic neural network. In 4th International IEEE/EMBS Conference on Neural Engineering. 367–370.
DOI: Google ScholarCross Ref - [4] . 2020. Wavelets for EEG Analysis. Retrieved from https://www.intechopen.com/chapters/74032.Google Scholar
- [5] . 2018. Comparison of machine learning methods for two class motor imagery tasks using EEG in brain-computer interface. In Advances in Science and Engineering Technology International Conferences (ASET’18). 1–5.
DOI: Google ScholarCross Ref - [6] . 2007. Classification methods for ongoing EEG and MEG signals. Biolog. Res. 40 (
00 2007), 415–437.DOI: Google ScholarCross Ref - [7] . 2015. Classification of human emotions from EEG signals using SVM and LDA classifiers. In 2nd International Conference on Signal Processing and Integrated Networks (SPIN’15). 180–185.
DOI: Google ScholarCross Ref - [8] . 2005. BCI Competition III. Retrieved from https://www.bbci.de/competition/iii/.Google Scholar
- [9] . 2020. Automatic epileptic seizure onset-offset detection based on CNN in scalp EEG. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’20). 1225–1229.
DOI: Google ScholarCross Ref - [10] . 2022. A prosthetic arm based on electroencephalography by signal acquisition and processing on MATLAB. Int. J. Res. Eng., Sci. Manag. 5, 1 (
Jan. 2022), 119–124. Retrieved from http://www.journals.resaim.com/ijresm/article/view/1691.Google Scholar - [11] . 2020. A unified framework and method for EEG-based early epileptic seizure detection and epilepsy diagnosis. IEEE Access 8 (2020), 20080–20092.
DOI: Google ScholarCross Ref - [12] . 2012. Dry and noncontact EEG sensors for mobile brain–computer interfaces. IEEE Trans. Neural Syst. Rehab. Eng. 20, 2 (2012), 228–235.
DOI: Google ScholarCross Ref - [13] . 2021. A brain-computer interface for controlling IoT devices using EEG signals. In IEEE 5th Ecuador Technical Chapters Meeting (ETCM’21). IEEE, 1–6.Google Scholar
- [14] . 2006. Using novel MEMS EEG sensors in detecting drowsiness application. In IEEE Biomedical Circuits and Systems Conference. 33–36.
DOI: Google ScholarCross Ref - [15] . 2014. Adolf Beck: A forgotten pioneer in electroencephalography. J. Hist. Neurosci. 23, 3 (2014), 276–286.
DOI: Google ScholarCross Ref - [16] . 2021. A deep neural network-based transfer learning to enhance the performance and learning speed of BCI systems. Brain-Comput. Interf. 8, 1-2 (2021), 14–25.Google ScholarCross Ref
- [17] . 2019. EEG-based biometrics utilizing image recognition for patient identification. In IEEE 10th Annual Ubiquitous Computing, Electronics Mobile Communication Conference (UEMCON’19). 0591–0595.
DOI: Google ScholarCross Ref - [18] . 2012. Online motor-imagery based BCI. In International Conference on Applied Electronics. 65–68.Google Scholar
- [19] . 2019. Emotion classification from EEG signals in convolutional neural networks. In Innovations in Intelligent Systems and Applications Conference (ASYU’19). 1–6.
DOI: Google ScholarCross Ref - [20] . 2022. Insight Brainwear 5 Channel Wireless EEG Headset | EMOTIV. Retrieved from https://www.emotiv.com/insight/.Google Scholar
- [21] . 2022. Application of dry EEG electrodes on low-cost SSVEP-based BCI for robot navigation. In IEEE International Conference on Imaging Systems and Techniques (IST’22). 1–6.
DOI: Google ScholarDigital Library - [22] . 2019. The dry revolution: Evaluation of three different EEG dry electrode types in terms of signal spectral features, mental states classification and usability. PubMed 19, 6 (2019), 1365.
DOI: Google ScholarCross Ref - [23] . 2007. A novel dry active electrode for EEG recording. IEEE Trans. Biomed. Eng. 54, 1 (2007), 162–165.
DOI: Google ScholarCross Ref - [24] . 2018. Decisions, goals, and actions. In Fundamentals of Cognitive Neuroscience (Second Edition), and (Eds.). Academic Press, San Diego, CA 279–319.
DOI: Google ScholarCross Ref - [25] . 2020. A novel bristle-shaped semi-dry electrode with low contact impedance and ease of use features for EEG signal measurements. IEEE Trans. Biomed. Eng. 67, 3 (2020), 750–761.
DOI: Google ScholarCross Ref - [26] . 2021. Intraoperative mapping of pre-central motor cortex and subcortex: A proposal for supplemental cortical and novel subcortical maps to Penfield’s motor homunculus. Brain Struct. Funct. 226, 5 (
01 June 2021), 1601–1611.DOI: Google ScholarCross Ref - [27] Hector Gonzalez, Richard George, Shahzad Muzaffar, Javier Acevedo, Sebastian Hoppner, Christian Mayr, Jerald Yoo, Frank Fitzek, and Ibrahim Elfadel. 2021. Hardware acceleration of EEG-based emotion classification systems: A comprehensive survey. IEEE Transactions on Biomedical Circuits and Systems 15, 3 (J2021), 412–442. Google ScholarCross Ref
- [28] Xiaotong Gu, Zehong Cao, Alireza Jolfaei, Peng Xu, Dongrui Wu, Tzyy-Ping Jung, and Chin-Teng Lin. 2021. EEG-based Brain-computer interfaces (BCIs): A survey of recent studies on signal sensing technologies and computational intelligence approaches and their applications. IEEE/ACM Transactions on Computational Biology and Bioinformatics 18, 5 (2021), 1645–1666. Google ScholarDigital Library
- [29] . 2017. Optimized deep learning for EEG big data and seizure prediction BCI via Internet of Things. IEEE Trans. Big Data 3, 4 (2017), 392–404.Google ScholarCross Ref
- [30] Rumeysa Ince, Saliha Seda Adanir, and Fatma Sevmez. 2021. The inventor of electroencephalography (EEG): Hans Berger (1873-1941). Child’s Nervous System 37, 9 (2021), 2723–2724. Google ScholarCross Ref
- [31] . 2020. Brain computer interface for neurorehabilitation with kinesthetic feedback. In 5th International Conference on Robotics and Automation Engineering (ICRAE’20). 153–157.
DOI: Google ScholarCross Ref - [32] . 2019. Wavelet transform based approach for EEG feature selection of motor imagery data for braincomputer interfaces. In 2019 3rd International Conference on Inventive Systems and Control (ICISC’19). 101–105.
DOI: Google ScholarCross Ref - [33] . 2014. Evaluation of the NeuroSky MindFlex EEG headset brain waves data. In IEEE 12th International Symposium on Applied Machine Intelligence and Informatics (SAMI’14). IEEE, 91–94.Google Scholar
- [34] . 2022. A transfer learning-based CNN and LSTM hybrid deep learning model to classify motor imagery EEG signals. Comput. Biol. Med. 143 (2022), 105288.
DOI: Google ScholarDigital Library - [35] . 2020. A comparative analysis of signal processing and classification methods for different applications based on EEG signals. Biocyber. Biomed. Eng. 40, 2 (2020), 649–690.Google ScholarCross Ref
- [36] . 2020. Accurate deep learning-based sleep staging in a clinical population with suspected obstructive sleep apnea. IEEE J. Biomed. Health Inform. 24, 7 (2020), 2073–2081.
DOI: Google ScholarCross Ref - [37] . 2021. BENDR: Using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data. Front. Hum. Neurosci. 15 (
06 2021), 653659.DOI: Google ScholarCross Ref - [38] . 2016. 3D printed dry EEG electrodes. Sensors 16, 10 (
Oct. 2016), 1635.DOI: Google ScholarCross Ref - [39] . 2012. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, , , , and (Eds.), Vol. 25. Curran Associates, Inc. Retrieved from https://proceedings.neurips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf.Google ScholarDigital Library
- [40] . 2004. Support vector channel selection in BCI. IEEE Trans. Biomed. Eng. 51, 6 (2004), 1003–1010.
DOI: Google ScholarCross Ref - [41] Wei Li, Wei Huan, Bowen Hou, Ye Tian, Zhen Zhang, and Aiguo Song. 2022. Can emotion be eransferred?—A review on transfer learning for EEG-based emotion recognition. IEEE Transactions on Cognitive and Developmental Systems 14, 3 (2022), 833–846. Google ScholarCross Ref
- [42] . 2023. Learning reliable neural networks with distributed architecture representations. ACM Trans. Asian Low-resour. Lang. Inf. Process. (
Jan. 2023).DOI: Google ScholarDigital Library - [43] . 2021. Convolutional neural network based electroencephalogram controlled robotic arm. In IEEE International Conference on Automatic Control Intelligent Systems (I2CACIS’21). 26–31.
DOI: Google ScholarCross Ref - [44] . 2014. Dry EEG electrodes. Sensors 14, 7 (
July 2014), 12847–12870.DOI: Google ScholarCross Ref - [45] . 2019. Microspike array electrode with flexible backing for biosignal monitoring. In IEEE SENSORS Conference. 1–4.
DOI: Google ScholarCross Ref - [46] . 2018. Scalable smart home interface using occipitalis sEMG detection and classification. In 9th IEEE Annual Ubiquitous Computing, Electronics Mobile Communication Conference (UEMCON’18). 1002–1008.
DOI: Google ScholarCross Ref - [47] . 2021. The classification of wink-based EEG signals: The identification of significant time-domain features. In Advances in Mechatronics, Manufacturing, and Mechanical Engineering. Springer, 283–291.Google ScholarCross Ref
- [48] . 2017. Optimal feature selection and deep learning ensembles method for emotion recognition from human brain EEG sensors. IEEE Access 5 (2017), 14797–14806.Google ScholarCross Ref
- [49] . 2007. Novel hybrid bioelectrodes for ambulatory zero-prep EEG measurements using multi-channel wireless EEG system. In Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 4565. 137–146.Google Scholar
- [50] . 2015. Emotion classification of EEG brain signal using SVM and KNN. In IEEE International Conference on Multimedia Expo Workshops (ICMEW’15). 1–5.
DOI: Google ScholarCross Ref - [51] . 2018. Study of the brain waves for the identification of the basic needs of patients with amyotrophic lateral sclerosis. In Congreso Argentino de Ciencias de la Informática y Desarrollos de Investigación (CACIDI’18). 1–6.
DOI: Google ScholarCross Ref - [52] . 2009. Automated detection of brain tumor in EEG signals using artificial neural networks. In International Conference on Advances in Computing, Control, and Telecommunication Technologies. 284–288.
DOI: Google ScholarDigital Library - [53] . 2016. Stacked probabilistic regularized LDA on partitioning non-stationary EEG data for left/right hand imagery classification. In IEEE EMBS Conference on Biomedical Engineering and Sciences (IECBES’16). 301–306.
DOI: Google ScholarCross Ref - [54] . 2021. Motor imagery classification based on a recurrent-convolutional architecture to control a hexapod robot. Mathematics 9, 6 (2021).
DOI: Google ScholarCross Ref - [55] . 2001. The five percent electrode system for high-resolution EEG and ERP measurements. Clin. Neurophysiol. 112, 4 (2001), 713–719.
DOI: Google ScholarCross Ref - [56] . 2021. OpenBCI Featured Products | OpenBCI Online Store. Retrieved from https://shop.openbci.com/collections/frontpage.Google Scholar
- [57] Debrupa Pal, Sujoy Palit, and Anilesh Dey. 2021. Brain computer interface: A review. Computational Advancement in Communication, Circuits and Systems (10 2021), 25–35. Google ScholarCross Ref
- [58] . 2021. Internet of Things and Access Control: Sensing, Monitoring and Controlling Access in IoT-Enabled Healthcare Systems. Vol. 37. Springer Nature.Google ScholarCross Ref
- [59] . 2020. Access control for Internet of Things—Enabled assistive technologies: An architecture, challenges and requirements. In Assistive Technology for the Elderly. Elsevier, 1–43.Google ScholarCross Ref
- [60] . 2018. Fine-grained access control for smart healthcare systems in the Internet of Things. EAI Endors. Trans. Industr. Netw. Intell. Syst. 4, 13 (2018).Google Scholar
- [61] . 2021. Development and progress in sensors and technologies for human emotion recognition. Sensors 21, 16 (2021), 5554.Google ScholarCross Ref
- [62] . 2023. An advanced healthcare system where internet of things meets brain-computer interface using event-related potential InInternational Conference on Distributed Computing and Networking. Association for Computing Machinery, New York, NY, 438–443.
DOI: Google ScholarDigital Library - [63] . 2022. Analysis Of EEG signals using open BCI to predict the stress level. In IEEE India Council International Subsections Conference (INDISCON’22). 1–6.
DOI: Google ScholarCross Ref - [64] . 2015. Brain–computer interface boosts motor imagery practice during stroke recovery. Ann. Neurol. 77, 5 (2015), 851–865.
DOI: Google ScholarCross Ref - [65] . 2009. Independent component analysis using clustering on motor imagery EEG. In Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 4735–4738.
DOI: Google ScholarCross Ref - [66] . 2020. Brain-computer interface based on motor imagery and emotion using convolutional neural networks. In FORTEI-International Conference on Electrical Engineering (FORTEI-ICEE’20). 108–112.
DOI: Google ScholarCross Ref - [67] . 2013. Brain–machine interface in chronic stroke rehabilitation: A controlled study. Ann. Neurol. 74, 1 (2013), 100–108.
DOI: Google ScholarCross Ref - [68] . 2021. A review of the role of machine learning techniques towards brain–computer interface applications. Mach. Learn. Knowl. Extract. 3, 4 (2021), 835–862.Google ScholarCross Ref
- [69] . 2023. Automatic epilepsy detection from EEG signals. In Proceedings of the 6th Joint International Conference on Data Science & Management of Data (CODS-COMAD’23). Association for Computing Machinery, New York, NY, 272–273.
DOI: Google ScholarDigital Library - [70] . 2022. Cross-correlated spectral entropy-based classification of EEG motor imagery signal for triggering lower limb exoskeleton. Sig., Image Vid. Process. (
02 Feb. 2022).DOI: Google ScholarCross Ref - [71] . 2019. Hybrid approach for classification of electroencephalographic signals using time–frequency images with wavelets and texture features. In Intelligent Data Analysis for Biomedical Applications, , , and (Eds.). Academic Press, 253–273.
DOI: Google ScholarCross Ref - [72] . 2014. VOG-enhanced ICA for SSVEP response detection from consumer-grade EEG. In 22nd European Signal Processing Conference (EUSIPCO’14). 2025–2029.Google Scholar
- [73] . 2018. Artificial neural networks to assess emotional states from brain-computer interface. Electronics 7, 12 (2018), 384.Google ScholarCross Ref
- [74] . 2003. The sense-think-act paradigm revisited. In 1st International Workshop on Robotic Sensing (ROSE’03). IEEE.Google Scholar
- [75] . 2015. Very Deep Convolutional Networks for Large-scale Image Recognition. Retrieved from https://arxiv.org/abs/1409.1556.Google Scholar
- [76] . 2017. Classification of motor imagery based EEG signals using sparsity approach. In International Conference on Intelligent Human Computer Interaction. 47–59.
DOI: Google ScholarCross Ref - [77] . 2022. Application of cognitive computing in healthcare, cybersecurity, big data and IoT: A literature review. Inf. Process. Manag. 59, 2 (2022), 102888.Google ScholarDigital Library
- [78] . 2009. One century of brain mapping using Brodmann areas. Clin. Neuroradiol. 19, 3 (
01 Aug. 2009), 179–186.DOI: Google ScholarCross Ref - [79] . 2020. EEG band separation using multilayer perceptron for efficient feature extraction and perfect BCI paradigm. In Emerging Technology in Computing, Communication and Electronics (ETCCE’20). 1–6.
DOI: Google ScholarCross Ref - [80] . 2018. Deep transfer learning for EEG-based brain computer interface. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’18). 916–920.
DOI: Google ScholarDigital Library - [81] . 2016. Power line and ocular artifact denoising from EEG using notch filter and wavelet transform. In 3rd International Conference on Computing for Sustainable Global Development (INDIACom’16). 1654–1659.Google Scholar
- [82] . 1993. Sex differences in age regression parameters of healthy adults-normative data and practical implications. Electroenceph. Clin. Neurophysiol. 86, 6 (1993), 377–384.
DOI: Google ScholarCross Ref - [83] Edmond Q. Wu, Zhengtao Cao, Pengwen Xiong, Aiguo Song, Li-Min Zhu, and Mengsun Yu. 2022. Brain-computer interface using brain power map and cognition detection network during flight. IEEE/ASME Transactions on Mechatronics 27, 5 (2022), 3942–3952. Google ScholarCross Ref
- [84] . 2018. Towards objective assessment of movie trailer quality using human electroencephalogram and facial recognition. In IEEE International Conference on Electro/Information Technology (EIT’18). 0449–0452.
DOI: Google ScholarCross Ref - [85] . 2019. Recognizing human emotion patterns by applying fast Fourier transform based on brainwave features. In International Conference on Informatics, Multimedia, Cyber and Information System (ICIMCIS’19). IEEE, 249–254.Google ScholarCross Ref
- [86] . 2021. A survey on robots controlled by motor imagery brain-computer interfaces. Cognit. Robot. 1 (2021), 12–24.Google ScholarCross Ref
- [87] . 2021. Hybrid deep neural network using transfer learning for EEG motor imagery decoding. Biomed. Sig. Process. Contr. 63 (2021), 102144.
DOI: Google ScholarCross Ref
Index Terms
- Multimodal Social Data Analytics on the Design and Implementation of an EEG-Mechatronic System Interface
Recommendations
Subject-Independent Motor Imagery EEG Classification Based on Graph Convolutional Network
Pattern RecognitionAbstractElectroencephalogram (EEG) motor imagery (MI) has attracted much attention in brain-computer interfaces (BCIs) as it directly encodes human intentions. However, the variability of EEG-based brain signals between individuals requires current BCI ...
A novel combination of time phase and EEG frequency components for SSVEP-Based BCI
ICONIP'11: Proceedings of the 18th international conference on Neural Information Processing - Volume Part IThe steady-state visual evoked potential (SSVEP) has been widely applied in brain-computer interfaces (BCIs), such as letter or icon selection and device control. Most of these BCIs used different flickering frequencies to evoke SSVEP with different ...
Bagging of EEG Signals for Brain Computer Interface
ITA '13: Proceedings of the 2013 International Conference on Information Technology and ApplicationsBrain-computer interfaces (BCIs) aim to provide a non-muscular channel to communicate with the external world through the use of the brain Electroencephalograph (EEG) activity. A crucial step in such an operation is brain signal processing methods. BCI ...
Comments