Brain–computer interface controlled gaming: Evaluation of usability by severely motor restricted end-users

Abstract

Objective

Connect-Four, a new sensorimotor rhythm (SMR) based brain–computer interface (BCI) gaming application, was evaluated by four severely motor restricted end-users; two were in the locked-in state and had unreliable eye-movement.

Methods

Following the user-centred approach, usability of the BCI prototype was evaluated in terms of effectiveness (accuracy), efficiency (information transfer rate (ITR) and subjective workload) and users’ satisfaction.

Results

Online performance varied strongly across users and sessions (median accuracy (%) of end-users: A = .65; B = .60; C = .47; D = .77). Our results thus yielded low to medium effectiveness in three end-users and high effectiveness in one end-user. Consequently, ITR was low (0.05–1.44 bits/min). Only two end-users were able to play the game in free-mode. Total workload was moderate but varied strongly across sessions. Main sources of workload were mental and temporal demand. Furthermore, frustration contributed to the subjective workload of two end-users. Nevertheless, most end-users accepted the BCI application well and rated satisfaction medium to high. Sources for dissatisfaction were (1) electrode gel and cap, (2) low effectiveness, (3) time-consuming adjustment and (4) not easy-to-use BCI equipment. All four end-users indicated ease of use as being one of the most important aspect of BCI.

Conclusion

Effectiveness and efficiency are lower as compared to applications using the event-related potential as input channel. Nevertheless, the SMR-BCI application was satisfactorily accepted by the end-users and two of four could imagine using the BCI application in their daily life. Thus, despite moderate effectiveness and efficiency BCIs might be an option when controlling an application for entertainment.

Introduction

Brain–computer interfaces (BCIs) provide muscle independent communication and control of the environment by circumventing motor pathways [1]. For more than 20 years BCIs have been investigated in healthy subjects and, albeit less, in subjects with different kinds of diseases, such as amyotrophic lateral sclerosis (ALS), spinal cord injury, multiple sclerosis, stroke or muscular dystrophy, with the goal to develop BCI devices that can be used in daily life. BCIs have been developed for communication [2], [3], [4], entertainment [5], [6], [7], e-inclusion [2], [8] and environmental control, e.g. [9].

BCIs are either controlled by regulation of a specific component of the electrical activity of the brain (electroencephalogram, EEG) or by event-related potentials (ERPs) that are elicited by sensory stimulation. Regulation of EEG activity can be achieved by neurofeedback, i.e. the user receives feedback of his or her brain activity in real-time within a closed-loop design. Neurofeedback and stimulation can be provided in the visual, auditory or tactile modality. The most commonly applied BCIs are those that use the P300-ERP (P300-BCI) and sensorimotor rhythm (SMR) activity as input signal for BCI control. SMR-BCIs are modulated by motor imagery (MI) [10]. Current improvements of stimulation in visual [11], [12], [13], and auditory P300-BCIs [14], [15], feature extraction approaches [16], and also machine learning approaches in SMR-BCIs [17] enhance accuracy, speed and information transfer rate (ITR) impressively, even in severely motor restricted end-users, e.g. [18].

BCI studies with potential end-users, i.e. patients with diseases leading to severe motor impairment, yielded high differences in performance, possibly due to high heterogeneity in paradigms and type of diseases, e.g. [2], [3], [4], [19], for reviews see [20], [21]. In a study of Kübler et al. [19], four severely motor restricted patients diagnosed with ALS were trained to control a cursor by regulating SMR. After 20 sessions of training, all four patients reached successful control (above 70% correct responses). Piccione et al. found that severely motor restricted patients were able to control a P300 based BCI [22]. The authors suggest a possible negative relationship between progress of disease and BCI performance. However, these results could not be confirmed by other studies, which used the P300-BCI with ALS patients for many sessions [3], [4]. Although this issue remains unresolved, end-users with disease proved to be able to control the P300-BCI and to regulate SMR, when provided with feedback and sufficient training time [19].

Despite an almost exponential increase in BCI studies [23], [24] for improving e.g. classifier and ITR, studies investigating the usability of BCI applications with potential end-users are sparse. To our knowledge, there are only few studies that systematically evaluated the usability of communication and e-inclusion [2] and entertainment BCI prototypes (see [25] same Special Issue) with end-users. In the first evaluation study of Zickler et al., four severely motor impaired possible end-users tested the BCI in which the P300 stimulation was superimposed on a standard, commercially distributed assistive technology (AT) software enabling the end-users to enter text, send emails or surf the internet. End-users indicated the low speed and effectiveness, long adjustment time and EEG cap/wet electrodes as the main obstacles of the BCI device and none of the end-users could imagine using the BCI in daily life unless substantially improved [2].

The current study aimed at evaluating the usability of a new SMR-BCI-controlled gaming prototype in terms of its effectiveness (accuracy), efficiency (ITR and subjective workload) and end-users’ satisfaction following a user-centred approach [2], [26]. It is well established that MI reliably modulates SMR in the alpha (9–12 Hz) and beta (13–30 Hz) frequency bands. With MI, e.g. imagined hand or feet movement, SMR desynchronizes over the respective area of sensorimotor cortices [10]. In the current study, four possible BCI end-users with severe motor impairment were trained to play the game “Connect-Four” by MI induced modulation of the SMR and to evaluate this gaming application. The user-centred approach was standardized with the ISO 9241-210 [27]. The user-centred process implies (i) understand the user, the task and environmental requirements, (ii) encourage early and active involvement of users, (iii) be driven and refined by user-centred evaluation, (iv) iterate developmental process of design solutions, (v) incorporate the whole user experience, (vi) encourage multi-disciplinary design (see also [25] same Special Issue). The goal of this user-centred evaluation process is to develop a new BCI application into a final product that matches the end-user's needs and requirements, that is accepted well by the end-users and can be used for entertainment in their daily life [2], [26], [28], [29].