Elsevier

Entertainment Computing

Volume 5, Issue 4, December 2014, Pages 391-399
Entertainment Computing

Comparing interaction techniques for serious games through brain–computer interfaces: A user perception evaluation study

https://doi.org/10.1016/j.entcom.2014.10.004Get rights and content

Highlights

  • Control in 3D a serious game using different BCIs via brainwaves.

  • Evaluate the effectiveness of different EEG technologies.

  • Compare the applicability of BCIs in serious games.

Abstract

This paper examines the application of commercial and non-invasive electroencephalography (EEG)-based brain–computer (BCIs) interfaces with serious games. Two different EEG-based BCI devices were used to fully control the same serious game. The first device (NeuroSky MindSet) uses only a single dry electrode and requires no calibration. The second device (Emotiv EPOC) uses 14 wet sensors requiring additional training of a classifier. User testing was performed on both devices with sixty-two participants measuring the player experience as well as key aspects of serious games, primarily learnability, satisfaction, performance and effort. Recorded feedback indicates that the current state of BCIs can be used in the future as alternative game interfaces after familiarisation and in some cases calibration. Comparative analysis showed significant differences between the two devices. The first device provides more satisfaction to the players whereas the second device is more effective in terms of adaptation and interaction with the serious game.

Introduction

The past decade has seen a huge proliferation of commercial interaction devices for video games. Each of these new devices offers a diverse way of interacting with games and computer generated simulations. Typical technologies that these devices use include: optical, auditory, magnetic and inertia sensors. Some can operate as autonomous controllers while others work in hybrid mode (with standard I/O devices such as mouse and keyboard). However, only the hybrid approaches appear to be functional, the rest require a lot of physical effort. This restricts users’ expressive capabilities as well as the information transferred from the user to the computer [1]. Nowadays, non-invasive brain–computer interfaces (BCIs) are getting a lot of attention as alternative human–computer interaction devices for games and virtual environments [2], [3].

Non-invasive BCIs operate by recording the brain activity from the scalp with Electroencephalography (EEG) sensors attached to the head on an electrode cap or headset without being surgically implanted. However, they still have a number of problems and they cannot function as accurately as other natural user interfaces (NUIs) and traditional input devices such as the standard keyboard and mouse [4]. The Information Transfer Rate (ITR) of this kind of BCIs is still around 25 bits per minute [5], which makes them much slower compared to traditional input devices such as keyboard (which have typical speed of over 300 characters per minute, roughly 400 bits per minute) [6]. The main reasons behind this are due to bad classification, long training procedures, latency issues and cumbersome hardware [7]. Also, because of lack of training and accessibility in using BCI devices, some people find it difficult to use at all [8].

The majority of BCI studies are performed in laboratory environments under controlled conditions. However this is not always possible in real-life applications and makes current BCI technology not quite suitable for practical applications and widespread use [9]. Game designers and researchers must make sure that BCIs used for gaming environments should not become a barrier in terms of the interaction [10] but on the contrary a more effective medium. Although non-invasive BCI technologies seem to have the potential of providing an environment where “thoughts are not constrained by what is physically possible” [7], they are still not ready for commercial use.

The main aim of the paper is to examine the effectiveness of two different BCI devices for fully controlling an avatar inside a serious game. The objectives of the paper are threefold. Firstly, to enable a user to fully control an avatar in real-time using only EEG data. Secondly, to qualitatively examine the behaviour and different reactions of the users while playing the game and, thirdly, to test each device in terms of: learnability of the interface using the game, satisfaction of the player, performance of the interfaces and effort expended by the player. Two different EEG-based BCI devices were used; one which requires no calibration (NeuroSky MindSet) and another one that requires the training of the classifiers (Emotiv EPOC). The user is visually stimulated by fully controlling an avatar in the serious game (see Section 3). Two different types of EEG-based BCIs were used: the NeuroSky MindSet and the Emotiv EPOC. All tests (N = 62) were conducted using the same serious game, which was integrated with the devices; participants were divided equally across the devices.

The rest of the paper is structured as follows. Section 2 provides background information for serious games and BCIs. Section 3, presents the serious game that was used as a case study called Roma Nova. Section 4 demonstrates how the two different BCI devices were used for controlling the same serious game. Finally, Section 5 presents the evaluation results and Section 6 the conclusions and future work.

Section snippets

Background

Early non-invasive BCI research methods for serious games interaction were usually oriented towards the medical domain rather than entertainment. This kind of research was targeting locked-in patients where haptic and linguistic interfaces fail. The first BCI game was created in 1977. In this game, the user could move in four directions in a maze by fixating on one of four diamond-shaped points periodically flashed. The methodology used was far ahead of its time using online artefact rejection

The Roma Nova serious game

The aim of the Rome Reborn project was to create highly realistic 3D representations illustrating the urban development of ancient Rome from the first settlement in the late Bronze Age (ca. 1000 B.C.) to the depopulation of the city in the early Middle Ages (ca. A.D. 550) [28]. The project includes hundreds of buildings, thirty-two of which are highly detailed monuments reconstructed on the basis of reliable archaeological evidence. The remainder of the 25–30 km2 model are filled with

BCI interactions

This section presents the two different BCI devices that were used (NeuroSky MindSet and the Emotiv), the hardware configuration as well as the methods used for interacting with the serious game.

User study

This section presents the procedure followed for the evaluation together with the qualitative and quantitative results of both studies as well as a comparative study.

Conclusions and future work

This paper presented two different ways of fully interacting with the same serious game using non-invasive BCIs. Two different EEG-based BCI devices were used, one single dry electrode which requires no calibration and a second that needs some calibration classifier training in order to create a user profile using also 14 wet electrodes. Overall the results indicate that both BCI technologies offer the potential of being used as alternative game interfaces prior to some familiarisation with the

Acknowledgements

The authors would like to thank the Human Computer Interaction Laboratory at Masaryk University, the WMG at the Warwick University as well as the Serious Games Institute (SGI) at Coventry University members for their support and inspiration. Two videos that demonstrate both systems in operation can be found online at: <http://www.youtube.com/watch?v=L6t4Ji5yu7k> and <http://www.youtube.com/watch?v=5Y_clGGoO4Y>.

References (36)

  • F. Lotte

    Brain–computer interfaces for 3D games: hype or hope?

  • P. Coulton et al.

    Brain interaction for mobile games

  • J.J. Vidal

    Real-time detection of brain events in EEG

    Proc. IEEE

    (1977)
  • L.F. Nicolas-Alonso et al.

    Brain computer interfaces, a review

    Sensors

    (2012)
  • L. Bi et al.

    EEG-based brain-controlled mobile robots: a survey

    IEEE Trans. Hum. Mach. Syst. IEEE Comput. Soc.

    (2013)
  • J.A. Pineda et al.

    Learning to control brain rhythms: making a brain–computer interface possible

    IEEE Trans. Neural Syst. Rehabil. Eng.

    (2003)
  • R. Krepki et al.

    The Berlin brain–computer interface (BBCI) – towards a new communication channel for online control in gaming applications

    Multimedia Tools Appl.

    (2007)
  • J.D. Bayliss

    Use of the evoked potential P3 component for control in a virtual apartment

    IEEE Trans. Neural Syst. Rehabil. Eng.

    (2003)
  • Cited by (25)

    View all citing articles on Scopus

    This paper has been recommended for acceptance by Christos Gatzidis, Ph.D.

    View full text