Abstract
We describe how the classic P300 based BCI-controlled spelling paradigm was transformed into a painting tool. Initiated by an artist, we (“we” refers to all people who were involved in the making of Brain Painting at any one time. “I”, “me” or “AK” refers to the first author. Individual names/initials are used to identify a specific person being involved in a specific step of the history including end-users with disease) developed Brain Painting in close interaction with two end-users of the technology in the locked-in state due to amyotrophic lateral sclerosis. We enabled them to use Brain Painting at their home independent of the researchers being present. Throughout the use of Brain Painting we applied measures of effectiveness, efficiency, and end-user satisfaction leading to a unique long-term evaluation covering hundreds of sessions. We demonstrated a positive effect on quality of life, self-esteem, and other measures of well-being. The artists created a plethora of Brain Paintings, which have been shown at several exhibitions. In this chapter, we describe firstly, the history of Brain Painting. Secondly, we explain in detail how the P300-BCI controlled application works. Thirdly, we dwell on how we enabled use of Brain Painting in daily life at the end-users’ home. We, fourthly, dedicate a separate section to the impact of Brain Painting on quality of life, and show finally some of the Brain Paintings and summarize the artists’ Brain Painting exhibitions. We end with a summary, emphasising that if BCI developers follow the user-centred design, i.e., listen to and interact closely with the targeted end-users of technology, the transfer of BCI-controlled applications out of the laboratory to the end-users’ home becomes more likely, feasible and hopefully more real.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
Initials of Brain Painting end-users with permission.
- 4.
- 5.
HHEM is Heide Pfützner’s artist name, so we use it instead of her initials.
- 6.
Standard deviation.
- 7.
The “f” stands for “faces” as in such paradigms an N400 is elicited in response to face perception (Eimer 2000).
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
References
Botrel L, Holz EM, Kübler A (2015) Brain Painting V2: evaluation of P300-based brain-computer interface for creative expression by an end-user following the user-centered design. Brain Comput Interfaces 2(2–3):135–149. https://doi.org/10.1080/2326263x.2015.1100038
Brady NC, Bruce S, Goldman A, Erickson K, Mineo B, Ogletree BT, Paul D, Romski MA, Sevcik R, Siegel E, Schoonover J, Snell M, Sylvester L, Wilkinson K (2016) Communication services and supports for individuals with severe disabilities: guidance for assessment and intervention. Am J Intellect Dev Disabil 121(2):121–138. https://doi.org/10.1352/1944-7558-121.2.121
Cheng J, Jin J, Wang X (2017) Comparison of the BCI performance between the semitransparent face pattern and the traditional face pattern. Comput Intell Neurosci 2017:1323985. https://doi.org/10.1155/2017/1323985
Demers L, Weiss-Lambrou R, Ska B (2002) The Quebec user evaluation of satisfaction with assistive technology (QUEST 2.0): an overview and recent progress. Technol Disabil 14(3):101–105
Eimer M (2000) Event-related brain potentials distinguish processing stages involved in face perception and recognition. Clin Neurophysiol 111(4):694–705
Farwell LA, Donchin E (1988) Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr Clin Neurophysiol 70(6):510–523
Funk M, Raschke M (2013) Brain Painting: action paintings based on BCI-input. University of Bremen, Paper presented at the Mensch & Computer
Guy V, Soriani MH, Bruno M, Papadopoulo T, Desnuelle C, Clerc M (2018) Brain computer interface with the P300 speller: usability for disabled people with amyotrophic lateral sclerosis. Ann Phys Rehabil Med 61(1):5–11. https://doi.org/10.1016/j.rehab.2017.09.004
Halder S, Kathner I, Kubler A (2016) Training leads to increased auditory brain-computer interface performance of end-users with motor impairments. Clin Neurophysiol 127(2):1288–1296. https://doi.org/10.1016/j.clinph.2015.08.007
Hoffmann U, Vesin JM, Ebrahimi T, Diserens K (2008) An efficient P300-based brain-computer interface for disabled subjects. J Neurosci Methods 167(1):115–125. https://doi.org/10.1016/j.jneumeth.2007.03.005
Holz EM, Kaufmann T, Franz D, Hösle A, Kübler A (2012) Brain drawing: first evaluation results. Paper presented at the TOBI workshop III, Würzburg, Germany
Holz EM, Botrel L, Kübler A (2014) Independent BCI use in two patients diagnosed with amyotrophic lateral sclerosis. Paper presented at the 6th international BCI conference, Graz, Austria, 16–19 September 2014
Holz EM, Botrel L, Kaufmann T, Kubler A (2015a) Long-term independent brain-computer interface home use improves quality of life of a patient in the locked-in state: a case study. Arch Phys Med Rehab 96(3 Suppl):S16–26. https://doi.org/10.1016/j.apmr.2014.03.035
Holz EM, Botrel L, Kübler A (2015b) Independent home use of Brain Painting improves quality of life of two artists in the locked-in state diagnosed with amyotrophic lateral sclerosis. Brain Comput Interfaces 2(2–3):117–134. https://doi.org/10.1080/2326263x.2015.1100048
ISO 9241-210 (2008) Ergonomics of human system interaction—Part 210: Human-centred design for interactive systems (formerly known as 13407). International Organization for Standardization (ISO), Switzerland
Jin J, Allison BZ, Kaufmann T, Kübler A, Zhang Y, Wang X, Cichocki A (2012) The changing face of P300 BCIs: a comparison of stimulus changes in a P300 BCI involving faces, emotion, and movement. PLoS ONE 7(11):e49688. https://doi.org/10.1371/journal.pone.0049688
Jin J, Allison BZ, Zhang Y, Wang X, Cichocki A (2014) An ERP-based BCI using an oddball paradigm with different faces and reduced errors in critical functions. Int J Neural Syst 24(8):1450027. https://doi.org/10.1142/s0129065714500270
Jutai J, Day H (2002) Psychosocial impact of assistive devices scale (PIADS). Technol Disabil 14:107–111
Kaufmann T, Schulz SM, Grunzinger C, Kübler A (2011) Flashing characters with famous faces improves ERP-based brain-computer interface performance. J Neural Eng 8(5):056016. https://doi.org/10.1088/1741-2560/8/5/056016
Kaufmann T, Völker S, Gunesch L, Kübler A (2012) Spelling is just a click away—a user-centered brain-computer interface including auto-calibration and predictive text entry. Front Neurosci 6:72. https://doi.org/10.3389/fnins.2012.00072
Kaufmann T, Schulz SM, Koblitz A, Renner G, Wessig C, Kübler A (2013) Face stimuli effectively prevent brain-computer interface inefficiency in patients with neurodegenerative disease. Clin Neurophysiol 124(5):893–900. https://doi.org/10.1016/j.clinph.2012.11.006
Kübler A, Neumann N, Kaiser J, Kotchoubey B, Hinterberger T, Birbaumer N (2001) Brain-computer communication: self-regulation of slow cortical potentials for verbal communication. Arch Phys Med Rehabil 82(11):1533–1539
Kübler A, Halder S, Furdea A, Hösle A (2008) Brain Painting: BCI meets art. Paper presented at the 4th international brain-computer interface workshop and training course, Graz, Austria
Kübler A, Holz EM, Riccio A, Zickler C, Kaufmann T, Kleih SC, Staiger-Sälzer P, Desideri L, Hoogerwerf EJ, Mattia D (2014) The user-centered design as novel perspective for evaluating the usability of BCI-controlled applications. PLoS ONE 9(12):e112392. https://doi.org/10.1371/journal.pone.0112392
Kübler A, Holz EM, Sellers EW, Vaughan TM (2015) Toward independent home use of brain-computer interfaces: a decision algorithm for selection of potential end-users. Arch Phys Med Rehabil 96(3 Suppl):S27–32. https://doi.org/10.1016/j.apmr.2014.03.036
Mc Cane LM, Sellers EW, McFarland DJ, Mak JN, Carmack CS, Zeitlin D, Wolpaw JR, Vaughan TM (2014) Brain-computer interface (BCI) evaluation in people with amyotrophic lateral sclerosis. Amyotroph Lateral Scler Frontotemporal Degener 15(3–4):207–215. https://doi.org/10.3109/21678421.2013.865750
Müller-Putz GR, Breitwieser C, Cincotti F, Leeb R, Schreuder M, Leotta F, Tavella M, Bianchi L, Kreilinger A, Ramsay A, Rohm M, Sagebaum M, Tonin L, Neuper C, del Millan RJ (2011) Tools for brain-computer interaction: a general concept for a hybrid BCI. Front Neuroinform 5:30. https://doi.org/10.3389/fninf.2011.00030
Münßinger JI, Halder S, Kleih SC, Furdea A, Raco V, Hösle A, Kübler A (2010) Brain Painting: first evaluation of a new brain-bomputer interface application with ALS-patients and healthy volunteers. Front Neurosci 4:182. https://doi.org/10.3389/fnins.2010.00182
Nijboer F (2015) Technology transfer of brain-computer interfaces as assistive technology: barriers and opportunities. Ann Phys Rehabil Med 58(1):35–38. https://doi.org/10.1016/j.rehab.2014.11.001
Nijboer F, Sellers EW, Mellinger J, Jordan MA, Matuz T, Furdea A, Halder S, Mochty U, Krusienski DJ, Vaughan TM, Wolpaw JR, Birbaumer N, Kubler A (2008) A P300-based brain-computer interface for people with amyotrophic lateral sclerosis. Clin Neurophysiol 119(8):1909–1916. https://doi.org/10.1016/j.clinph.2008.03.034
Riccio A, Holz EM, Aricò P, Leotta F, Aloise F, Desideri L, Rimondini M, Kübler A, Mattia D, Cincotti F (2015) Hybrid P300-based brain-computer interface to improve usability for people with severe motor disability: electromyographic signals for error correction during a spelling task. Arch Phys Med Rehabil 96(3 Suppl):S54–S61. https://doi.org/10.1016/j.apmr.2014.05.029
Ryan DB, Colwell KA, Throckmorton CS, Collins LM, Caves K, Sellers EW (2018) Evaluating brain-computer interface performance in an ALS population: checkerboard and color paradigms. Clin EEG Neurosci 49(2):114–121. https://doi.org/10.1177/1550059417737443
Sellers EW, Vaughan TM, Wolpaw JR (2010) A brain-computer interface for long-term independent home use. Amyotroph Lateral Scler 11(5):449–455
Severens M, Van der Waal M, Farquhar J, Desain P (2014) Comparing tactile and visual gaze-independent brain-computer interfaces in patients with amyotrophic lateral sclerosis and healthy users. Clin Neurophysiol. https://doi.org/10.1016/j.clinph.2014.03.005
van de Laar B, Brugman I, Nijboer F, Poel M, Nijholt A (2013) BrainBrush, a multimodal application for creative expressivity. Paper presented at the the sixth international conference on advances in computer-human interactions, Nice, France, 24 February–1 March
Wolpaw JR, Bedlack RS, Reda DJ, Ringer RJ, Banks PG, Vaughan TM, Heckman SM, McCane LM, Carmack CS, Winden S, McFarland DJ, Sellers EW, Shi H, Paine T, Higgins DS, Lo AC, Patwa HS, Hill KJ, Huang GD, Ruff RL (2018) Independent home use of a brain-computer interface by people with amyotrophic lateral sclerosis. Neurology 91(3):e258–e267. https://doi.org/10.1212/WNL.0000000000005812
Yeom SK, Fazli S, Muller KR, Lee SW (2014) An efficient ERP-based brain-computer interface using random set presentation and face familiarity. PLoS ONE 9(11):e111157. https://doi.org/10.1371/journal.pone.0111157
Zickler C, Di Donna V, Kaiser V, Al-Khodairy A, Kleih S, Kübler A, Malavasi M, Mattia D, Mongardi S, Neuper C, Rohm M, Rupp R, Staiger-Sälzer P, Hoogerwerf E-J (2009) BCI applications for people with disabilities: defining user needs and user requirements. Paper presented at the 10th association of the advancement of assistive technology in Europe conference, Florence, Italy
Zickler C, Halder S, Kleih SC, Herbert C, Kübler A (2013) Brain Painting: usability testing according to the user-centered design in end users with severe motor paralysis. Artif Intell Med 59(2):99–110. https://doi.org/10.1016/j.artmed.2013.08.003
Acknowledgements
We thank all the patient and healthy end-users who participated in short- and long-term studies throughout the years. Thanks to Dr. Elisa Holz, Dr. Sonja Kleih, Dr. Femke Nijboer, Dipl.-Psych. Claudia Zickler, Dr. Adrian Furdea, Dr. Sebastian Halder who contributed to the making of Brain Painting. Dr. Elisa Holz together with Dr. Loïc Botrel conducted the long-term studies with HHEM and JT. We thank Adi Hösle for the idea, his endeavour and contribution to the making of Brain Painting. This work was supported by the European ICT Program ProjectFP7-224631 (TOBI). This manuscript only reflects the authors’ views and funding agencies are not liable for any use that may be made of the information contained herein. HHEM is funded by gtec, Linz, Austria (EEG cap and electrodes).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Kübler, A., Botrel, L. (2019). The Making of Brain Painting—From the Idea to Daily Life Use by People in the Locked-in State. In: Nijholt, A. (eds) Brain Art. Springer, Cham. https://doi.org/10.1007/978-3-030-14323-7_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-14323-7_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-14322-0
Online ISBN: 978-3-030-14323-7
eBook Packages: Computer ScienceComputer Science (R0)