Skip to main content
SpringerLink
Log in

Different effects of using pictures as stimuli in a P300 brain-computer interface under rapid serial visual presentation or row-column paradigm

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Previous proposals for controlling a P300-based BCI speller have shown an improvement using alternative images instead of letters as target stimuli under a row-column paradigm (RCP). However, the RCP is not suitable for those patients with a lack of gaze control. To solve that, the rapid serial visual presentation (RSVP) paradigm has been proposed in previous studies. The aim of the present work is to assess if a set of alternative pictures that improved performance in RCP could also improve performance in RSVP. Sixteen participants controlled four conditions in calibration and online tasks: letters in RCP, pictures in RCP, letters in RSVP and pictures in RSVP. The effect given by pictures was greater under RCP than under RSVP, both for performance and event-related potential analyses. Indeed, pictures did not show any improvement under RSVP in comparison to letters. In addition, the condition with pictures under RCP was declared the favourite by most users (68.75%), while the condition with pictures under RSVP was not chosen as favourite by any participant. Therefore, this work shows that the improvement related to the use of pictures as alternative flashing stimuli under RCP may not be transferred to RSVP.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Tao G, Charm G, Kabacińska K, Miller WC, Robillard JM (2020) Evaluation tools for assistive technologies: a scoping review. Arch Phys Med Rehabil 101:1025–1040. https://doi.org/10.1016/j.apmr.2020.01.008

    Article  PubMed  Google Scholar 

  2. Patterson JR, Grabois M (1986) Locked-in syndrome: a review of 139 cases. Stroke 17:758–764. https://doi.org/10.1161/01.STR.17.4.758

    Article  CAS  PubMed  Google Scholar 

  3. Bauer G, Gerstenbrand F, Rumpl E (1979) Varieties of the locked-in syndrome. J Neurol 221:77–91. https://doi.org/10.1007/BF00313105

    Article  CAS  PubMed  Google Scholar 

  4. Murguialday AR, Hill J, Bensch M, Martens S, Halder S, Nijboer F, Schoelkopf B, Birbaumer N, Gharabaghi A (2011) Transition from the locked in to the completely locked-in state: a physiological analysis. Clin Neurophysiol 122:925–933. https://doi.org/10.1016/j.clinph.2010.08.019

    Article  PubMed  Google Scholar 

  5. Fuchino Y, Nagao M, Katura T, Bando M, Naito M, Maki A, Nakamura K, Hayashi H, Koizumi H, Yoro T (2008) High cognitive function of an ALS patient in the totally locked-in state. Neurosci Lett 435:85–89. https://doi.org/10.1016/j.neulet.2008.01.046

    Article  CAS  PubMed  Google Scholar 

  6. Lemoignan J, Ells C (2010) Amyotrophic lateral sclerosis and assisted ventilation: how patients decide. Palliat Support Care 8:207–213. https://doi.org/10.1017/S1478951510000027

    Article  PubMed  Google Scholar 

  7. Wolpaw JR, Birbaumer N, McFarland DJ et al (2002) Brain-computer interfaces for communication and control. Clin Neurophysiol 113:767–791. https://doi.org/10.1016/S1388-2457(02)00057-3

    Article  PubMed  Google Scholar 

  8. Birbaumer N (2006) Breaking the silence: brain-computer interfaces (BCI) for communication and motor control. Psychophysiology 43:517–532. https://doi.org/10.1111/j.1469-8986.2006.00456.x

    Article  PubMed  Google Scholar 

  9. Farwell LA, Donchin E (1988) Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr Clin Neurophysiol 70:510–523. https://doi.org/10.1016/0013-4694(88)90149-6

    Article  CAS  PubMed  Google Scholar 

  10. Zhang R, Wang Q, Li K, He S, Qin S, Feng Z, Chen Y, Song P, Yang T, Zhang Y, Yu Z, Hu Y, Shao M, Li Y (2017) A BCI-based environmental control system for patients with severe spinal cord injuries. IEEE Trans Biomed Eng 64:1959–1971

    Article  Google Scholar 

  11. Aydın EA, Bay ÖF, Güler İ (2016) Implementation of an embedded web server application for wireless control of brain computer interface based home environments. J Med Syst 40:1–10. https://doi.org/10.1007/s10916-015-0386-0

    Article  Google Scholar 

  12. Rezeika A, Benda M, Stawicki P, Gembler F, Saboor A, Volosyak I (2018) Brain–computer interface spellers: a review. Brain Sci 8:57. https://doi.org/10.3390/brainsci8040057

    Article  PubMed Central  Google Scholar 

  13. Sutton S, Braren M, Zubin J, John ER (1965) Evoked-potential correlates of stimulus uncertainty. Science 150:1187–1188. https://doi.org/10.1126/science.150.3700.1187

    Article  CAS  PubMed  Google Scholar 

  14. Brunner P, Joshi S, Briskin S, Wolpaw JR, Bischof H, Schalk G (2010) Does the “P300” speller depend on eye gaze? J Neural Eng 7:56013. https://doi.org/10.1088/1741-2560/7/5/056013

    Article  CAS  Google Scholar 

  15. Treder MS, Blankertz B (2010) (C)overt attention and visual speller design in an ERP-based brain-computer interface. Behav Brain Funct 6:28. https://doi.org/10.1186/1744-9081-6-28

    Article  PubMed  PubMed Central  Google Scholar 

  16. Acqualagna L, Treder MS, Schreuder M, Blankertz B (2010) A novel brain-computer interface based on the rapid serial visual presentation paradigm. In: 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC’10. pp 2686–2689

  17. Acqualagna L, Treder MS, Blankertz B (2013) Chroma speller: isotropic visual stimuli for truly gaze-independent spelling. Int IEEE/EMBS Conf Neural Eng NER 1041–1044. doi:https://doi.org/10.1109/NER.2013.6696115

  18. Aricò P, Aloise F, Schettini F et al (2011) GeoSpell: an alternative P300-based speller interface towards no eye gaze required. Int J Bioelectromagn 13:152–153

    Google Scholar 

  19. Sato H, Washizawa Y (2016) An N100-P300 spelling brain-computer interface with detection of intentional control. Computers 5:31. https://doi.org/10.3390/computers5040031

    Article  Google Scholar 

  20. Sellers EW, Krusienski DJ, McFarland DJ et al (2006) A P300 event-related potential brain-computer interface (BCI): the effects of matrix size and inter stimulus interval on performance. Biol Psychol 73:242–252. https://doi.org/10.1016/j.biopsycho.2006.04.007

    Article  PubMed  Google Scholar 

  21. Ryan DB, Frye GE, Townsend G, Berry DR, Mesa-G S, Gates NA, Sellers EW (2011) Predictive spelling with a P300-based brain-computer interface: increasing the rate of communication. Int J Hum Comput Interact 27:69–84. https://doi.org/10.1080/10447318.2011.535754

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Townsend G, LaPallo BK, Boulay CB et al (2010) A novel P300-based brain-computer interface stimulus presentation paradigm: moving beyond rows and columns. Clin Neurophysiol 121:1109–1120. https://doi.org/10.1016/j.clinph.2010.01.030

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Kaufmann T, Schulz SM, Grünzinger C, Kübler A (2011) Flashing characters with famous faces improves ERP-based brain-computer interface performance. J Neural Eng 8:056016. https://doi.org/10.1088/1741-2560/8/5/056016

    Article  CAS  PubMed  Google Scholar 

  24. Fernández-Rodríguez Á, Medina-Juliá MT, Velasco-Álvarez F, Ron-Angevin R (2020) Effects of spatial stimulus overlap in a visual P300-based brain-computer interface. Neuroscience 431:134–142. https://doi.org/10.1016/j.neuroscience.2020.02.011

    Article  CAS  PubMed  Google Scholar 

  25. Li Q, Liu S, Li J, Bai O (2015) Use of a green familiar faces paradigm improves P300-speller brain-computer interface performance. PLoS One 10:1–15. https://doi.org/10.1371/journal.pone.0130325

    Article  CAS  Google Scholar 

  26. Fernández-Rodríguez Á, Velasco-Álvarez F, Medina-Juliá MT, Ron-Angevin R (2019) Evaluation of emotional and neutral pictures as flashing stimuli using a P300 brain-computer interface speller. J Neural Eng 16:056024. https://doi.org/10.1088/1741-2552/ab386d

    Article  PubMed  Google Scholar 

  27. Fernández-Rodríguez Á, Medina-Juliá MTMT, Velasco-Álvarez F, Ron-Angevin R (2019) Preliminary results using a P300 brain-computer interface speller: a possible interaction effect between presentation paradigm and set of stimuli. In: Rojas I, Joya G, Catala A (eds) Advances in Computational Intelligence. Springer International Publishing, pp 371–381

  28. Schalk G, McFarland DJ, Hinterberger T et al (2004) BCI2000: a general-purpose brain-computer interface (BCI) system. IEEE Trans Biomed Eng 51:1034–1043

    Article  Google Scholar 

  29. Velasco-Álvarez F, Sancha-Ros S, García-Garaluz E, Fernández-Rodríguez Á, Medina-Juliá MT, Ron-Angevin R (2019) UMA-BCI Speller: an easily configurable P300 speller tool for end users. Comput Methods Prog Biomed 172:127–138. https://doi.org/10.1016/J.CMPB.2019.02.015

    Article  Google Scholar 

  30. Cheng J, Jin J, Wang X (2017) Comparison of the BCI performance between the semitransparent face pattern and the traditional face pattern. Comput Intell Neurosci 2017:1323985–1323989. https://doi.org/10.1155/2017/1323985

    Article  PubMed  PubMed Central  Google Scholar 

  31. Lang PJ, Bradley MM, Cuthbert BN (2008) International affective picture system (IAPS): affective ratings of pictures and instruction manual. Technical Report A-8. University of Florida, Gainesville, FL

  32. Wolpaw JR, Ramoser H, McFarland DJ, Pfurtscheller G (1998) EEG-based communication: improved accuracy by response verification. IEEE Trans Rehabil Eng 6:326–333. https://doi.org/10.1109/86.712231

    Article  CAS  PubMed  Google Scholar 

  33. Chennu S, Alsufyani A, Filetti M, Owen AM, Bowman H (2013) The cost of space independence in P300-BCI spellers. J Neuroeng Rehabil 10:1–13. https://doi.org/10.1186/1743-0003-10-82

    Article  Google Scholar 

  34. Ma Z, Qiu T (2017) Performance improvement of ERP-based brain–computer interface via varied geometric patterns. Med Biol Eng Comput 55:2245–2256. https://doi.org/10.1007/s11517-017-1671-5

    Article  PubMed  Google Scholar 

  35. Delorme A, Makeig S (2004) EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods 134:9–21. https://doi.org/10.1016/j.jneumeth.2003.10.009

    Article  Google Scholar 

  36. Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Ser B 57:289–300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x

    Article  Google Scholar 

  37. Lu Z, Li Q, Gao N, Yang J (2020) The self-face paradigm improves the performance of the P300-speller system. Front Comput Neurosci 13:1–12. https://doi.org/10.3389/fncom.2019.00093

    Article  Google Scholar 

  38. Li S, Jin J, Daly I, Zuo C, Wang X, Cichocki A (2020) Comparison of the ERP-based BCI performance among chromatic (RGB) semitransparent face patterns. Front Neurosci 14:1–12. https://doi.org/10.3389/fnins.2020.00054

    Article  CAS  Google Scholar 

  39. Jones MR, Sellers EW (2019) Faces, locations, and tools: a proposed two-stimulus P300 brain computer interface. J Neural Eng 16:1–17. https://doi.org/10.1088/1741-2552/aaff22

    Article  Google Scholar 

  40. Kellicut-Jones MR, Sellers EW (2018) P300 brain-computer interface: comparing faces to size matched non-face stimuli. Brain-Computer Interfaces 5:30–39. https://doi.org/10.1080/2326263X.2018.1433776

    Article  Google Scholar 

  41. Kaufmann T, Kübler A (2014) Beyond maximum speed - a novel two-stimulus paradigm for brain-computer interfaces based on event-related potentials (P300-BCI). J Neural Eng 11:056004. https://doi.org/10.1088/1741-2560/11/5/056004

    Article  PubMed  Google Scholar 

  42. Kutas M, Sereno MI, Ganis G (1996) The search for “common sense”: an electrophysiological study of the comprehension of words and pictures in reading. J Cogn Neurosci 8:89–106

    Article  Google Scholar 

  43. Garrison KE, Crowell AL, Finley AJ, Schmeichel BJ (2017) Effects of prior mental effort on picture processing: an ERP investigation. Psychophysiology 54:1714–1725. https://doi.org/10.1111/psyp.12914

    Article  PubMed  Google Scholar 

  44. Aloise F, Aricò P, Schettini F, Riccio A, Salinari S, Mattia D, Babiloni F, Cincotti F (2012) A covert attention P300-based brain-computer interface: Geospell. Ergonomics 55:538–551. https://doi.org/10.1080/00140139.2012.661084

    Article  PubMed  Google Scholar 

  45. Pires G, Nunes U, Castelo-Branco M (2011) GIBS block speller: toward a gaze-independent P300-based BCI. Proc Annu Int Conf IEEE Eng Med Biol Soc EMBS 6360–6364. doi:https://doi.org/10.1109/IEMBS.2011.6091570

  46. Ron-Angevin R, Varona-Moya S, Da Silva-Sauer L (2015) Initial test of a T9-like P300-based speller by an ALS patient. J Neural Eng 12:046023. https://doi.org/10.1088/1741-2560/12/4/046023

    Article  CAS  PubMed  Google Scholar 

  47. Guger C, Spataro R, Allison BZ, Heilinger A, Ortner R, Cho W, la Bella V (2017) Complete locked-in and locked-in patients: Command following assessment and communication with vibro-tactile P300 and motor imagery brain-computer interface tools. Front Neurosci 11:251. https://doi.org/10.3389/fnins.2017.00251

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

Moreover, the authors would like to thank M.J. Blanca-Mena for her support in statistical analyses and all participants for their cooperation.

Funding

This work was partially supported by the project SICCAU: RTI2018-100912-B-I00 (MCIU/AEI/FEDER, UE).

Author information

Authors and Affiliations

  1. Departamento de Tecnología Electrónica, Universidad de Málaga, 35 Louis Pasteur Boulevard, 29071, Malaga, Spain

    Álvaro Fernández-Rodríguez, María Teresa Medina-Juliá, Francisco Velasco-Álvarez & Ricardo Ron-Angevin

Authors
  1. Álvaro Fernández-Rodríguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. María Teresa Medina-Juliá
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Francisco Velasco-Álvarez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ricardo Ron-Angevin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Álvaro Fernández-Rodríguez.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fernández-Rodríguez, Á., Medina-Juliá, M.T., Velasco-Álvarez, F. et al. Different effects of using pictures as stimuli in a P300 brain-computer interface under rapid serial visual presentation or row-column paradigm. Med Biol Eng Comput 59, 869–881 (2021). https://doi.org/10.1007/s11517-021-02340-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-021-02340-y

Keywords

Search

Navigation