Abstract
Brain-computer interface (BCI) is a communication system that can help lock-in patients to interact with the outside environment by translating brain signals into machine commands. The present work provides a design for a virtual reality (VR) based BCI system that allows human participants to control a virtual hand to make gestures by P300 signals, with a positive peak of potential about 300 ms posterior to the onset of target stimulus. In this virtual environment, the participants can obtain a more immersed experience with the BCI system, such as controlling a virtual hand or walking around in the virtual world. Methods of modeling the virtual hand and analyzing the P300 signals are also described in detail. Template matching and support vector machine were used as the P300 classifier and the experiment results showed that both algorithms perform well in the system. After a short time of practice, most participants could learn to control the virtual hand during the online experiment with greater than 70% accuracy.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bayliss, J.D., Inverso, S.A., Tentler, A., 2004. Changing the P300 brain computer interface. Cyberpsych. & Behav., 7(6):694–704. [doi:10.1089/cpb.2004.7.694]
Beverina, F., Palmas, G., Silvoni, S., Piccione, F., 2003. User adaptive BCIs SSVEP and P300 based interfaces. PsychNol. J., 1(4):331–354.
Boser, B.E., Guyon, I.M., Vapnik, V., 1992. A Training Algorithm for Optimal Margin Classifiers. Proc. 5th Annual Workshop on Computational Learning Theory, p.144–152. [doi:10.1145/130385.130401]
Cortes, C., Vapnik, U., 1995. Support-vector networks. Mach. Learn., 20(3):273–297. [doi:10.1023/A:1022627411411]
Croft, R.J., Barry, R.J., 2000. Removal of ocular artifact from the EEG: a review. Neurophys. Clin., 30(1):5–19. [doi:10.1016/S0987-7053(00)00055-1]
Farwell, L.A., Donchin, E., 1988. Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephal. Clin. Neurophys., 70(6):510–523. [doi:10.1016/0013-4694(88)90149-6]
Feng, Z.Y., Chen, W.D., Ye, X.S., Zhang, S.M., Zheng, X.J., Wang, P., Jiang, J., Jin, L., Xu, Z.J., Liu, C.Q., et al., 2007. A remote control training system for rat navigation in complicated environment. J. Zhejiang Univ. Sci. A, 8(2):323–330. [doi:10.1631/jzus.2007.A0323]
Finke, A., Lenhardt, A., Ritter, H., 2009. The MindGame: a P300-based brain computer interface game. Neur. Networks, 22(9):1329–1333. [doi:10.1016/j.neunet.2009.07.003]
Friedman, D., Leeb, R., Guger, C., Steed, A., Pfurtscheller, G., 2007. Navigating virtual reality by thought: what is it like? Pres. Teleoper. Virt. Environ., 16(1):100–110. [doi:10.1162/pres.16.1.100]
Hoffmann, U., Vesin, J.M., Ebrahimi, T., Diserens, K., 2008. An efficient P300-based brain-computer interface for disabled subjects. J. Neuros. Methods, 167(1):115–125. [doi:10.1016/j.jneumeth.2007.03.005]
Krusienski, D.J., Sellers, E.W., McFarland, D.J., Vaughan, T.M., Wolpaw, J.R., 2008. Toward enhanced P300 speller performance. J. Neurosci. Methods, 167(1):15–21. [doi:10.1016/j.jneumeth.2007.07.017]
Lebedev, M.A., Nicolelis, M.A., 2006. Brain-machine interfaces: past, present and future. Trends Neurosci., 29(9):536–546. [doi:10.1016/j.tins.2006.07.004]
Lee, J., Kunii, T.L., 1995. Model-based analysis of hand posture. IEEE Comput. Graph. Appl., 15(5):77–86. [doi:10.1109/38.403831]
Li, Y., Zhang, J.H., Su, Y., Chen, W.D., Qi, Y., Zhang, J.C., Zheng, X.X., 2009. P300 Based BCI Messenger. IEEE/ICME Int. Conf. on Complex Medical Engineering, p.1–5. [doi:10.1109/ICCME.2009.4906636]
Lin, J., Wu, Y., Huang, T.S., 2000. Modeling the Constraints of Human Hand Motion. Proc. Workshop on Human Motion, p.121–126. [doi:10.1109/HUMO.2000.897381]
Nijboer, F., Sellers, E.W., Mellinger, J., Jordan, M.A., Matuz, T., Furdea, A., Halder, S., Mochty, U., Krusienski, D.J., Vaughan, T.M., et al., 2008. A P300-based brain-computer interface for people with amyotrophic lateral sclerosis. Clin. Neurophys., 119(8):1909–1916. [doi:10.1016/j.clinph.2008.03.034]
Piccione, F., Giorgi, F., Tonin, P., Priftis, K., Giove, S., Silvoni, S., Palmas, G., Beverina, F., 2006. A P300-based brain computer interface reliability and performance in healthy and paralysed participants. Clin. Neurophys., 117(3):531–547. [doi:10.1016/j.clinph.2005.07.024]
Piccione, F., Priftis, K., Tonin, P., 2008. Task and stimulation paradigm effects in a P300 brain computer interface exploitable in a virtual environment: a pilot study. PsychNol. J., 6(1):99–108.
Sellers, E.W., Donchin, E., 2006. A P300-based brain-computer interface: initial tests by ALS patients. J. Neurosci. Methods, 117:538–548. [doi:10.1016/j.clinph.2005.06.027]
Sellers, E.W., Krusienski, D.J., McFarland, D.J., Vaughan, T.M., Wolpaw, J.R., 2006. A P300 event-related potential brain-computer interface (BCI): the effects of matrix size and inter stimulus interval on performance. Biol. Psychol., 73(3):242–252. [doi:10.1016/j.biopsycho.2006.04.007]
Serby, H., Yom-Tov, E., Inbar, G.F., 2005. An improved P300-based brain-computer interface. IEEE Trans. Neur. Syst. Rehabil. Eng., 13(1):89–98. [doi:10.1109/TNSRE.2004.841878]
Sturman, D.J., 1992. Whole-Hand Input. PhD Thesis, Massachusetts Institute of Technology, USA.
Su, Y., Wu, B., Chen, W.D., 2008. P300-based brain computer interface: prototype of a Chinese speller. J. Comput. Inform. Syst., 4(4):1515–1522.
Theodoridis, S., Koutroumbas, K., 2006. Pattern Recognition. Elsevier/Academic Press, Boston.
Vaughan, T.M., Wolpaw, J.R., 2006. The third international meeting on brain-computer interface technology: making a difference. IEEE Trans. Neur. Syst. Rehabil. Eng., 14(2):126–127. [doi:10.1109/TNSRE.2006.875649]
Vaughan, T.M., Heetderks, W.J., Trejo, L.J., Rymer, W.Z., Weinrich, M., Moore, M.M., Kübler, A., Dobkin, B.H., Birbaumer, N., Donchin, E., et al., 2003. Brain-computer interface technology: a review of the second international meeting. IEEE Trans. Neur. Syst. Rehabil. Eng., 11(2):94–109. [doi:10.1109/TNSRE.2003.814799]
Wolpaw, J.R., Birbaumer, N., Heetderks, W.J., McFarland, D.J., Peckham, P.H., Schalk, G., Donchin, E., Quatrano, L.A., Robinson, C.J., Vaughan, T.M., 2000. Brain-computer interface technology: a review of the first international meeting. IEEE Trans. Rehabil. Eng., 8(2):164–173. [doi:10.1109/TRE.2000.847807]
Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., Vaughan, T.M., 2002. Brain-computer interfaces for communication and control. Clin. Neurophys., 113(6): 767–791. [doi:10.1016/S1388-2457(02)00057-3]
Yi, B., Harris, F.C.Jr., Wang, L., Yan, Y., 2005. Real-time natural hand gestures. Comput. Sci. Eng., 7(3):92–96. [doi:10.1109/MCSE.2005.58]
Zhang, H.H., Guan, C.T., Wang, C.C., 2008. Asynchronous P300-based brain-computer interfaces: a computational approach with statistical models. IEEE Trans. Biomed. Eng., 55(6):1754–1763. [doi:10.1109/TBME.2008.919128]
Zhao, Q.B., Zhang, L.Q., Cichocki, A., 2009. EEG-based asynchronous BCI control of a car in 3D virtual reality environments. Chin. Sci. Bull., 54(1):78–87. [doi:10.1007/s11434-008-0547-3]
Author information
Authors and Affiliations
Corresponding author
Additional information
Project supported by the National Natural Science Foundation of China (No. 60873125), the National Institute of Biomedical Imaging and Bioengineering (No. 1R03EB008235-01A1), the Shanghai Commission of Science and Technology (No. 10440710200), and the Fundamental Research Funds for the Central Universities
Rights and permissions
About this article
Cite this article
Chen, Wd., Zhang, Jh., Zhang, Jc. et al. A P300 based online brain-computer interface system for virtual hand control. J. Zhejiang Univ. - Sci. C 11, 587–597 (2010). https://doi.org/10.1631/jzus.C0910530
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/jzus.C0910530
Key words
- Brain-computer interface (BCI)
- Electroencephalography (EEG)
- P300
- Virtual reality (VR)
- Template matching
- Support vector machine (SVM)