Comparing interaction techniques for serious games through brain–computer interfaces: A user perception evaluation study☆
Introduction
The past decade has seen a huge proliferation of commercial interaction devices for video games. Each of these new devices offers a diverse way of interacting with games and computer generated simulations. Typical technologies that these devices use include: optical, auditory, magnetic and inertia sensors. Some can operate as autonomous controllers while others work in hybrid mode (with standard I/O devices such as mouse and keyboard). However, only the hybrid approaches appear to be functional, the rest require a lot of physical effort. This restricts users’ expressive capabilities as well as the information transferred from the user to the computer [1]. Nowadays, non-invasive brain–computer interfaces (BCIs) are getting a lot of attention as alternative human–computer interaction devices for games and virtual environments [2], [3].
Non-invasive BCIs operate by recording the brain activity from the scalp with Electroencephalography (EEG) sensors attached to the head on an electrode cap or headset without being surgically implanted. However, they still have a number of problems and they cannot function as accurately as other natural user interfaces (NUIs) and traditional input devices such as the standard keyboard and mouse [4]. The Information Transfer Rate (ITR) of this kind of BCIs is still around 25 bits per minute [5], which makes them much slower compared to traditional input devices such as keyboard (which have typical speed of over 300 characters per minute, roughly 400 bits per minute) [6]. The main reasons behind this are due to bad classification, long training procedures, latency issues and cumbersome hardware [7]. Also, because of lack of training and accessibility in using BCI devices, some people find it difficult to use at all [8].
The majority of BCI studies are performed in laboratory environments under controlled conditions. However this is not always possible in real-life applications and makes current BCI technology not quite suitable for practical applications and widespread use [9]. Game designers and researchers must make sure that BCIs used for gaming environments should not become a barrier in terms of the interaction [10] but on the contrary a more effective medium. Although non-invasive BCI technologies seem to have the potential of providing an environment where “thoughts are not constrained by what is physically possible” [7], they are still not ready for commercial use.
The main aim of the paper is to examine the effectiveness of two different BCI devices for fully controlling an avatar inside a serious game. The objectives of the paper are threefold. Firstly, to enable a user to fully control an avatar in real-time using only EEG data. Secondly, to qualitatively examine the behaviour and different reactions of the users while playing the game and, thirdly, to test each device in terms of: learnability of the interface using the game, satisfaction of the player, performance of the interfaces and effort expended by the player. Two different EEG-based BCI devices were used; one which requires no calibration (NeuroSky MindSet) and another one that requires the training of the classifiers (Emotiv EPOC). The user is visually stimulated by fully controlling an avatar in the serious game (see Section 3). Two different types of EEG-based BCIs were used: the NeuroSky MindSet and the Emotiv EPOC. All tests (N = 62) were conducted using the same serious game, which was integrated with the devices; participants were divided equally across the devices.
The rest of the paper is structured as follows. Section 2 provides background information for serious games and BCIs. Section 3, presents the serious game that was used as a case study called Roma Nova. Section 4 demonstrates how the two different BCI devices were used for controlling the same serious game. Finally, Section 5 presents the evaluation results and Section 6 the conclusions and future work.
Section snippets
Background
Early non-invasive BCI research methods for serious games interaction were usually oriented towards the medical domain rather than entertainment. This kind of research was targeting locked-in patients where haptic and linguistic interfaces fail. The first BCI game was created in 1977. In this game, the user could move in four directions in a maze by fixating on one of four diamond-shaped points periodically flashed. The methodology used was far ahead of its time using online artefact rejection
The Roma Nova serious game
The aim of the Rome Reborn project was to create highly realistic 3D representations illustrating the urban development of ancient Rome from the first settlement in the late Bronze Age (ca. 1000 B.C.) to the depopulation of the city in the early Middle Ages (ca. A.D. 550) [28]. The project includes hundreds of buildings, thirty-two of which are highly detailed monuments reconstructed on the basis of reliable archaeological evidence. The remainder of the 25–30 km2 model are filled with
BCI interactions
This section presents the two different BCI devices that were used (NeuroSky MindSet and the Emotiv), the hardware configuration as well as the methods used for interacting with the serious game.
User study
This section presents the procedure followed for the evaluation together with the qualitative and quantitative results of both studies as well as a comparative study.
Conclusions and future work
This paper presented two different ways of fully interacting with the same serious game using non-invasive BCIs. Two different EEG-based BCI devices were used, one single dry electrode which requires no calibration and a second that needs some calibration classifier training in order to create a user profile using also 14 wet electrodes. Overall the results indicate that both BCI technologies offer the potential of being used as alternative game interfaces prior to some familiarisation with the
Acknowledgements
The authors would like to thank the Human Computer Interaction Laboratory at Masaryk University, the WMG at the Warwick University as well as the Serious Games Institute (SGI) at Coventry University members for their support and inspiration. Two videos that demonstrate both systems in operation can be found online at: <http://www.youtube.com/watch?v=L6t4Ji5yu7k> and <http://www.youtube.com/watch?v=5Y_clGGoO4Y>.
References (36)
- et al.
Turning shortcomings into challenges: brain–computer interfaces for games
Entertainment Comput.
(2009) - et al.
Brain–computer interfaces for communication and control
Clin. Neurophysiol.
(2002) - et al.
The MindGame: a P300-based brain–computer interface game
Neural Netw.
(2009) - et al.
Neurofeedback-based motor imagery training for brain–computer interface (BCI)
Neurosci. Methods Elsevier
(2009) - et al.
Modality switching and performance in a thought and speech controlled computer game
- et al.
Brain–computer interfaces, virtual reality and video games
IEEE Comput.
(2008) - et al.
Improving BCI performance after classification
- et al.
Joy Elements of Information Theory
(2006) - et al.
Human–computer interaction for BCI games: usability and user experience
- et al.
Brain–computer interfacing for intelligent systems
IEEE Intell. Syst. IEEE Comput. Soc.
(2008)
Brain–computer interfaces for 3D games: hype or hope?
Brain interaction for mobile games
Real-time detection of brain events in EEG
Proc. IEEE
Brain computer interfaces, a review
Sensors
EEG-based brain-controlled mobile robots: a survey
IEEE Trans. Hum. Mach. Syst. IEEE Comput. Soc.
Learning to control brain rhythms: making a brain–computer interface possible
IEEE Trans. Neural Syst. Rehabil. Eng.
The Berlin brain–computer interface (BBCI) – towards a new communication channel for online control in gaming applications
Multimedia Tools Appl.
Use of the evoked potential P3 component for control in a virtual apartment
IEEE Trans. Neural Syst. Rehabil. Eng.
Cited by (25)
The influence of graphical elements on user's attention and control on a neurofeedback-based game
2019, Entertainment ComputingCitation Excerpt :However, subjects enjoyed the game and were motivated to play it, as shown by the rating distributions in Fig. 9. Other authors in previous works also reported that subjects were motivated and enjoyed to play games using brain-computer interfaces, even tough the accuracy may be significantly lower in comparison to other common interaction devices, such as mouse and keyboard [17]. This may be related to the novelty effect caused by the BCI platform, which can also occur in other untraditional interaction paradigms, such as gestural interfaces [22,36].
Closing the loop – The human role in artificial intelligence for education
2022, Frontiers in PsychologyA Scorewriter Application using Electrooculography-based Human-Computer Interface
2022, 2022 17th International Conference on Control, Automation, Robotics and Vision, ICARCV 2022A novel approach for designing authentication system using a picture based P300 speller
2021, Cognitive NeurodynamicsBiofeedback Methods in Entertainment Video Games: A Review of Physiological Interaction Techniques
2021, Proceedings of the ACM on Human-Computer Interaction
- ☆
This paper has been recommended for acceptance by Christos Gatzidis, Ph.D.