Skip to main content
SpringerLink
Log in

Development and Assessment of a Self-paced BCI-VR Paradigm Using Multimodal Stimulation and Adaptive Performance

  • Conference paper
  • First Online:
Physiological Computing Systems (PhyCS 2016, PhyCS 2017, PhyCS 2018)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10057))

Abstract

Motor-Imagery based Brain-Computer Interfaces (BCIs) can provide alternative communication pathways to neurologically impaired patients. The combination of BCIs and Virtual Reality (VR) can provide induced illusions of movement to patients with low-level of motor control during motor rehabilitation tasks. Unfortunately, current BCI systems lack reliability and good performance levels in comparison with other types of computer interfaces. To date, there is little evidence on how BCI-based motor training needs to be designed for transferring rehabilitation improvements to real life. Based on our previous work, we showcase the development and assessment of NeuRow, a novel multiplatform immersive VR environment that makes use of multimodal stimulation through vision, sound and vibrotactile feedback and delivered through a VR Head Mounted Display. In addition, we integrated the Adaptive Performance Engine (APE), a statistical approach to optimize user control in a self-paced BCI-VR paradigm. In this paper, we describe the development and pilot assessment of NeuRow as well as its integration and assessment with APE.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Hanakawa, T.: Organizing motor imageries. Neurosci. Res. 1, 56–63 (2015)

    Google Scholar 

  2. Eaves, D.L., Haythornthwaite, L., Vogt, S.: Motor imagery during action observation modulates automatic imitation effects in rhythmical actions. Front. Hum. Neurosci. 8, 28 (2014)

    Article  Google Scholar 

  3. Neuper, C., Scherer, R., Wriessnegger, S., Pfurtscheller, G.: Motor imagery and action observation: modulation of sensorimotor brain rhythms during mental control of a brain–computer interface. Clin. Neurophysiol. 120, 239–247 (2009)

    Article  Google Scholar 

  4. Mulder, T.: Motor imagery and action observation: cognitive tools for rehabilitation. J. Neural Transm. 114, 1265–1278 (2007)

    Article  Google Scholar 

  5. Lledo, P.-M., Alonso, M., Grubb, M.S.: Adult neurogenesis and functional plasticity in neuronal circuits. Nat. Rev. Neurosci. 7, 179–193 (2006)

    Article  Google Scholar 

  6. Rossini, P.M., Calautti, C., Pauri, F., Baron, J.-C.: Post-stroke plastic reorganisation in the adult brain. Lancet Neurol. 2, 493–502 (2003)

    Article  Google Scholar 

  7. Dickstein, R., et al.: Effects of integrated motor imagery practice on gait of individuals with chronic stroke: a half-crossover randomized study. Arch. Phys. Med. Rehabil. 94, 2119–2125 (2013)

    Article  Google Scholar 

  8. Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., Vaughan, T.M.: Brain-Computer Interfaces for communication and control. Clin. Neurophysiol. Off. J. Int. Fed. Clin. Neurophysiol. 113, 767–791 (2002)

    Article  Google Scholar 

  9. Dobkin, B.H.: Brain-computer interface technology as a tool to augment plasticity and outcomes for neurological rehabilitation. J. Physiol. 579, 637–642 (2007)

    Article  Google Scholar 

  10. Lotte, F., et al.: Combining BCI with virtual reality: towards new applications and improved BCI. In: Allison, B., Dunne, S., Leeb, R., Del, R., Millán, J., Nijholt, A. (eds.) Towards Practical Brain-Computer Interface. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-29746-5_10

    Chapter  Google Scholar 

  11. Vourvopoulos, A., Niforatos, E., Hlinka, M., Škola, F., Liarokapis, F.: Investigating the effect of user profile during training for BCI-based games. In: 2017 9th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games), pp. 117–124 (2017)

    Google Scholar 

  12. Friedman, D.: Brain-computer interfacing and virtual reality. In: Nakatsu, R., Rauterberg, M., Ciancarini, P. (eds.) Handbook of Digital Games and Entertainment Technologies, pp. 1–22. Springer, Singapore (2015). https://doi.org/10.1007/978-981-4560-52-8_2-1

    Chapter  Google Scholar 

  13. Slater, M.: Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philos. Trans. R. Soc. Lond. B Biol. Sci. 364, 3549–3557 (2009)

    Article  Google Scholar 

  14. Bermudez i Badia, S., Cameirao, M.S.: The Neurorehabilitation Training Toolkit (NTT): a novel worldwide accessible motor training approach for at-home rehabilitation after stroke. Stroke Res. Treat. 2012 (2012). https://doi.org/10.1155/2012/802157

    Article  Google Scholar 

  15. Lange, B., et al.: Designing informed game-based rehabilitation tasks leveraging advances in virtual reality. Disabil. Rehabil. 34, 1863–1870 (2012)

    Article  Google Scholar 

  16. Trompetto, C., et al.: Pathophysiology of spasticity: implications for neurorehabilitation. BioMed Res. Int. 2014 (2014). https://doi.org/10.1155/2014/354906

    Article  Google Scholar 

  17. Vourvopoulos, A., Bermúdez i Badia, S.: Usability and cost-effectiveness in brain-computer interaction: is it user throughput or technology related? In: Proceedings of the 7th Augmented Human International Conference, Geneva, Switzerland. ACM (2016)

    Google Scholar 

  18. Allison, B.Z., Neuper, C.: Could anyone use a BCI? In: Tan, D.S., Nijholt, A. (eds.) Brain-Computer Interfaces, pp. 35–54. Springer, London (2010). https://doi.org/10.1007/978-1-84996-272-8_3

    Chapter  Google Scholar 

  19. Vidaurre, C., Blankertz, B.: Towards a cure for BCI illiteracy. Brain Topogr. 23, 194–198 (2009)

    Article  Google Scholar 

  20. Gomez-Rodriguez, M., Peters, J., Hill, J., Schölkopf, B., Gharabaghi, A., Grosse-Wentrup, M.: Closing the sensorimotor loop: haptic feedback facilitates decoding of motor imagery. J. Neural Eng. 8, 036005 (2011)

    Article  Google Scholar 

  21. Hinterberger, T., et al.: A multimodal brain-based feedback and communication system. Exp. Brain Res. 154, 521–526 (2004)

    Article  Google Scholar 

  22. Leeb, R., Gwak, K., Kim, D.-S., del R Millán, J.: Freeing the visual channel by exploiting vibrotactile BCI feedback. In: Conference Proceedings: Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society Annual Conference 2013, pp. 3093–3096 (2013)

    Google Scholar 

  23. Cincotti, F., et al.: Vibrotactile feedback for brain-computer interface operation. Comput. Intell. Neurosci. 2007, e48937 (2007)

    Article  Google Scholar 

  24. Gwak, K., Leeb, R., Millan, J.D.R., Kim, D.-S.: Quantification and reduction of visual load during BCI operation. In: 2014 IEEE International Conference on Systems, Man and Cybernetics (SMC), pp. 2795–2800 (2014)

    Google Scholar 

  25. Jeunet, C., Vi, C., Spelmezan, D., N’Kaoua, B., Lotte, F., Subramanian, S.: Continuous tactile feedback for motor-imagery based brain-computer interaction in a multitasking context. In: Abascal, J., Barbosa, S., Fetter, M., Gross, T., Palanque, P., Winckler, M. (eds.) INTERACT 2015. LNCS, vol. 9296, pp. 488–505. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-22701-6_36

    Chapter  Google Scholar 

  26. Yao, L., Meng, J., Zhang, D., Sheng, X., Zhu, X.: Combining motor imagery with selective sensation toward a hybrid-modality BCI. IEEE Trans. Biomed. Eng. 61, 2304–2312 (2014)

    Article  Google Scholar 

  27. Leonardis, D., Frisoli, A., Solazzi, M., Bergamasco, M.: Illusory perception of arm movement induced by visuo-proprioceptive sensory stimulation and controlled by motor imagery. In: 2012 IEEE Haptics Symposium (HAPTICS), pp. 421–424 (2012)

    Google Scholar 

  28. Vourvopoulos, A., Bermúdez i Badia, S.: Motor priming in virtual reality can augment motor-imagery training efficacy in restorative brain-computer interaction: a within-subject analysis. J. NeuroEng. Rehabil. 13, 69 (2016)

    Article  Google Scholar 

  29. Lotte, F.: On the need for alternative feedback training approaches for BCI. Presented at the Berlin Brain-Computer Interface Workshop September (2012)

    Google Scholar 

  30. Lotte, F., Larrue, F., Mühl, C.: Flaws in current human training protocols for spontaneous Brain-Computer Interfaces: lessons learned from instructional design. Front. Hum. Neurosci. 7, 568 (2013)

    Article  Google Scholar 

  31. Shute, V.J.: Focus on formative feedback. Rev. Educ. Res. 78, 153–189 (2008)

    Article  Google Scholar 

  32. Hattie, J., Timperley, H.: The power of feedback. Rev. Educ. Res. 77, 81–112 (2007)

    Article  Google Scholar 

  33. Vourvopoulos, A., Ferreira, A., Bermúdez i Badia, S.: NeuRow: an immersive VR environment for motor-imagery training with the use of Brain-Computer Interfaces and vibrotactile feedback. Presented at the PhyCS 2016 - 3rd International Conference on Physiological Computing Systems Lisbon, July 2016

    Google Scholar 

  34. Ferreira, A., Vourvopoulos, A., Bermúdez i Badia, S.: Optimizing performance of non-expert users in brain-computer interaction by means of an adaptive performance engine. In: Guo, Y., Friston, K., Aldo, F., Hill, S., Peng, H. (eds.) BIH 2015. LNCS (LNAI), vol. 9250, pp. 202–211. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23344-4_20

    Chapter  Google Scholar 

  35. Klem, G.H., Lüders, H.O., Jasper, H.H., Elger, C.: The ten-twenty electrode system of the International Federation. The International Federation of Clinical Neurophysiology. Electroencephalogr. Clin. Neurophysiol. Suppl. 52, 3–6 (1999)

    Google Scholar 

  36. Renard, Y., et al.: OpenViBE: an open-source software platform to design, test, and use Brain-Computer Interfaces in real and virtual environments. Presence Teleoperators Virtual Environ. 19, 35–53 (2010)

    Article  Google Scholar 

  37. Vourvopoulos, A., Faria, A.L., Cameirao, M.S., Bermudez i Badia, S.: RehabNet: a distributed architecture for motor and cognitive neuro-rehabilitation. In: 2013 IEEE 15th International Conference on e-Health Networking, Applications Services (Healthcom), pp. 454–459 (2013)

    Google Scholar 

  38. Taylor II, R.M., Hudson, T.C., Seeger, A., Weber, H., Juliano, J., Helser, A.T.: VRPN: a device-independent, network-transparent VR peripheral system. In: Proceedings of the ACM Symposium on Virtual Reality Software and Technology, pp. 55–61. ACM, New York (2001)

    Google Scholar 

  39. Pfurtscheller, G., et al.: Graz-BCI: state of the art and clinical applications. IEEE Trans. Neural. Syst. Rehabil. Eng. Publ. IEEE Eng. Med. Biol. Soc. 11, 177–180 (2003)

    Google Scholar 

  40. Roberts, R., Callow, N., Hardy, L., Markland, D., Bringer, J.: Movement imagery ability: development and assessment of a revised version of the vividness of movement imagery questionnaire. J. Sport Exerc. Psychol. 30, 200–221 (2008)

    Article  Google Scholar 

  41. Adams, E., Ip, B.: From casual to core: a statistical mechanism for studying gamer dedication. http://www.gamasutra.com/view/feature/131397/from_casual_to_core_a_statistical_.php

  42. Hart, S.G., Staveland, L.E.: Development of NASA-TLX (task load index): results of empirical and theoretical research. In: Hancock, P.A., Meshkati, N. (eds.) Advances in Psychology, pp. 139–183. North-Holland, Amsterdam (1988)

    Google Scholar 

  43. IJsselsteijn, W., Poels, K., de Kort, Y.A.: The Game Experience Questionnaire: development of a self-report measure to assess player experiences of digital games. TU Eindh,. Eindhoven, Netherlands (2008)

    Google Scholar 

  44. Brooke, J.: SUS-A quick and dirty usability scale. Usability Eval. Ind. 189, 194 (1996)

    Google Scholar 

  45. Delorme, A., Makeig, S.: EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 134, 9–21 (2004)

    Article  Google Scholar 

  46. Pope, A.T., Bogart, E.H., Bartolome, D.S.: Biocybernetic system evaluates indices of operator engagement in automated task. Biol. Psychol. 40, 187–195 (1995)

    Article  Google Scholar 

  47. Berka, C., et al.: EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. Aviat. Space Environ. Med. 78, B231–B244 (2007)

    Google Scholar 

  48. Boostani, R., Moradi, M.H.: A new approach in the BCI research based on fractal dimension as feature and Adaboost as classifier. J. Neural Eng. 1, 212–217 (2004)

    Article  Google Scholar 

  49. Garcia, G.N., Ebrahimi, T., Vesin, J.: Support vector EEG classification in the Fourier and time-frequency correlation domains. In: First International IEEE EMBS Conference on Neural Engineering. Conference Proceedings, pp. 591–594 (2003)

    Google Scholar 

  50. Obermaier, B., Guger, C., Neuper, C., Pfurtscheller, G.: Hidden Markov models for online classification of single trial EEG data. Pattern Recogn. Lett. 22, 1299–1309 (2001)

    Article  Google Scholar 

  51. Solhjoo, S., Moradi, M.: Mental task recognition: a comparison between some of classification methods. In: BIOSIGNAL 2004 International EURASIP Conference, pp. 24–26 (2004)

    Google Scholar 

  52. Bangor, A., Kortum, P., Miller, J.: Determining what individual SUS scores mean: adding an adjective rating scale. J Usability Stud. 4, 114–123 (2009)

    Google Scholar 

  53. Gregg, M., Hall, C., Butler, A.: The MIQ-RS: a suitable option for examining movement imagery ability. Evid. Based Complement. Altern. Med. ECAM. 7, 249–257 (2010)

    Article  Google Scholar 

  54. Sigrist, R., Rauter, G., Riener, R., Wolf, P.: Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review. Psychon. Bull. Rev. 20, 21–53 (2013)

    Article  Google Scholar 

  55. Neuper, C., Scherer, R., Reiner, M., Pfurtscheller, G.: Imagery of motor actions: differential effects of kinesthetic and visual-motor mode of imagery in single-trial EEG. Brain Res. Cogn. Brain Res. 25, 668–677 (2005)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the European Commission through the RehabNet project - Neuroscience Based Interactive Systems for Motor Rehabilitation - EC (303891 RehabNet FP7-PEOPLE-2011-CIG), by the MACBIOIDI project financed by the EC through the INTERREG program (MAC/1.1.b/098), by the Fundação para a Ciência e Tecnologia (Portuguese Foundation for Science and Technology) through SFRH/BD/97117/2013, and LARSyS (Laboratório de Robótica e Sistemas em Engenharia e Ciência) through UID/EEA/50009/2013.

Author information

Authors and Affiliations

  1. Faculdade das Ciências Exatas e da Engenharia, Universidade da Madeira, Campus Universitário da Penteada, 9020-105, Funchal, Portugal

    Athanasios Vourvopoulos, André Ferreira & Sergi Bermudez i Badia

  2. Madeira Interactive Technologies Institute, Polo Científico e Tecnológico da Madeira, Caminho da Penteada, 9020-105, Funchal, Portugal

    Athanasios Vourvopoulos & Sergi Bermudez i Badia

  3. University of Southern California, Los Angeles, CA, USA

    Athanasios Vourvopoulos

Authors
  1. Athanasios Vourvopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. André Ferreira
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sergi Bermudez i Badia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Athanasios Vourvopoulos .

Editor information

Editors and Affiliations

  1. Medical University of Graz, Graz, Austria

    Andreas Holzinger

  2. NASA Langley Research Center, Hampton, VA, USA

    Alan Pope

  3. IT - Instituto de Telecomunicações, Lisbon, Portugal

    Hugo Plácido da Silva

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Vourvopoulos, A., Ferreira, A., Bermudez i Badia, S. (2019). Development and Assessment of a Self-paced BCI-VR Paradigm Using Multimodal Stimulation and Adaptive Performance. In: Holzinger, A., Pope, A., Plácido da Silva, H. (eds) Physiological Computing Systems. PhyCS PhyCS PhyCS 2016 2017 2018. Lecture Notes in Computer Science(), vol 10057. Springer, Cham. https://doi.org/10.1007/978-3-030-27950-9_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-27950-9_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-27949-3

  • Online ISBN: 978-3-030-27950-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation