Abstract
It has become a reality for paralyzed patients to use independent BMI-based assistive application to facilitate their lives. However, it was still a problem to switch among applications, if applications like typing and wheelchair-control were integrated in a device. In order to solve this problem, this paper realized a virtual mouse by combining eye tracker and MI, hoping to help patients to switch BMI applications independently and conveniently. The improved VT filter proposed in this paper solved the stable validity problem of the cursor. During the 2000 ms for MI, the time when the mouse stabilized in the valid area of the file can reach 94%. Asynchronous MI was realized in this paper through sliding window to enhance the flexibility of the system. The classification accuracy of left hand and right hand was 92.36%, that of left hand and idle state was 90.28%, and that of right hand and idle state was 90.63%. The final results were obtained by voting, which reached 91.33%. The results proved that the decoding of MI data would not be impacted during the control of eye tracker, which meant that the parallel control mechanism toward multi-cognitive modality proposed in this paper was feasible. This multi-cognitive modality based parallel control mechanism is in line with cognitive habits of human beings, and promising in improving the performance of human-machine fusion systems, in the future.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Berger, H.I.: Über das Elektrenkephalogramm des Menschen. Eur. Arch. Psychiatry Clin. Neurosci. 87, 527–570 (1928)
Liu, Y., Habibnezhad, M., Jebelli, H.: Brain-computer interface for hands-free teleoperation of construction robots. Autom. Constr. 123 (2021)
B., et al.: Noninvasive neuroimaging enhances continuous neural tracking for robotic device control. Sci. Robot. 4(31), eaaw6844 (2019)
Han, X., et al.: A novel system of SSVEP-based human-robot coordination. J. Neural Eng. 16(1), 016006 (2019)
Kuhner, D., et al.: A service assistant combining autonomous robotics, flexible goal formulation, and deep-learning-based brain–computer interfacing. Robot. Auton. Syst. 116, 98–113 (2019)
Xu, Y., et al.: Shared control of a robotic arm using non-invasive brain–computer interface and computer vision guidance. Robot. Auton. Syst. 115, 121–129 (2019)
Hochberg, L.R., et al.: Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 485(7398), 372–375 (2012)
Wessberg, J., et al.: Real-time prediction of hand trajectory by ensembles of cortical neurons in primates. Nature 408(6810), 361–365 (2000)
Farwell, L.A., Donchin, E.: Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr. Clin. Neurophysiol. 70(6) (1988)
Serruya, M.D., et al.: Instant neural control of a movement signal. Nature (2002)
Kim, S.P., et al.: Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia. J. Neural Eng. 5(4), 455–476 (2008)
Simeral, J.D., et al.: Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array. J. Neural Eng. 8(2), 025027 (2011)
Degenhart, A.D., et al.: Stabilization of a brain–computer interface via the alignment of low-dimensional spaces of neural activity. Nature Biomed. Eng. 4(7), 1–14 (2020)
Egovnik, T., et al.: An analysis of the suitability of a low-cost eye tracker for assessing the cognitive load of drivers. Appl. Ergon. 68, 1–11 (2018)
Fabio, R.A., et al.: Longitudinal cognitive rehabilitation applied with eye-tracker for patients with Rett Syndrome. Res. Dev. Disabil. 111(1), 103891 (2021)
Daniela, V., et al.: Visual exploration patterns of human figures in action: an eye tracker study with art paintings. Front. Psychol. 6, 1636 (2015)
Chen, Y., Tsai, M.J.: Eye-hand coordination strategies during active video game playing: an eye-tracking study. Comput. Hum. Behav. 51(OCT.), 8–14 (2015)
Tchalenko, J.: Eye-hand coordination in portrait drawing (2001)
Gonzalez, D.A., Niechwiej-Szwedo, E.: The effects of monocular viewing on hand-eye coordination during sequential grasping and placing movements. Vis. Res. 128, 30–38 (2016)
Wang, H., et al.: Hybrid gaze/EEG brain computer interface for robot arm control on a pick and place task. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (2015)
Dong, X., et al.: Hybrid Brain Computer Interface via Bayesian integration of EEG and eye gaze. In: 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER) (2015)
Doherty, D.O., et al.: Exploring gaze-motor imagery hybrid brain-computer interface design. In: The IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (2015)
Olsen, A.: The Tobii I-VT fixation filter: algorithm description (2012)
Delorme, A., Makeig, S.: EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 134(1), 9–21 (2004)
Wu, Z., Huang, N.E.: Ensemble empirical mode decomposition: a noise-assisted data analysis method. Adv. Adapt. Data Anal. 1(1) (2009)
Huang, N.E., et al.: The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proc. Roy. Soc. Lond. Ser. A: Math. Phys. Eng. Sci. 454(1971), 903–995 (1998)
Müller-Gerking, J., Pfurtscheller, G., Flyvbjerg, H.: Designing optimal spatial filters for single-trial EEG classification in a movement task. Clin. Neurophysiol. 110(5), 787–798 (1999)
Ramoser, H., Muller-Gerking, J.: Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Trans. Rehabil. Eng. 8, 441–446 (2000)
Johnson, G.D., Krusienski, D.J.: Ensemble SWLDA classifiers for the P300 speller. In: Jacko, J.A. (ed.) HCI 2009. LNCS, vol. 5611, pp. 551–557. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-02577-8_60
Krusienski, D.J., et al.: Toward enhanced P300 speller performance. J. Neurosci. Methods 167(1), 15–21 (2008)
Zeyl, T., et al.: Adding real-time Bayesian ranks to error-related potential scores improves error detection and auto-correction in a P300 speller. IEEE Trans. Neural Syst. Rehabil. Eng. 24(1), 46–56 (2016)
Behroozmand, R., Korzyukov, O., Larson, C.R.: Effects of voice harmonic complexity on ERP responses to pitch-shifted auditory feedback. Clin. Neurophysiol. 122(12), 2408–2417 (2011)
Kim, K., et al.: Odor habituation can modulate very early olfactory event-related potential. Sci. Rep. 10(1), 18117 (2020)
Acknowledgement
This work was supported by the National Natural Science Foundation of China (62006239).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Ye, Z., Liu, Y., Yu, Y., Zeng, L., Zhou, Z., Xie, F. (2021). A Virtual Mouse Based on Parallel Cooperation of Eye Tracker and Motor Imagery. In: Peng, Y., Hu, SM., Gabbouj, M., Zhou, K., Elad, M., Xu, K. (eds) Image and Graphics. ICIG 2021. Lecture Notes in Computer Science(), vol 12890. Springer, Cham. https://doi.org/10.1007/978-3-030-87361-5_53
Download citation
DOI: https://doi.org/10.1007/978-3-030-87361-5_53
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87360-8
Online ISBN: 978-3-030-87361-5
eBook Packages: Computer ScienceComputer Science (R0)