A EOG-based switch and its application for “start/stop” control of a wheelchair
Introduction
Human–computer interfaces (HCIs), which enable users to communicate with various external devices, are becoming increasingly indispensable in daily life [1]. There have been marked efforts in recent years to augment or integrate traditional HCIs (like a keyboard and mouse) by using biological signals, e.g., electroencephalography (EEG) [2] and electrooculography (EOG) [3]. EEG- and EOG-based HCIs allow people who suffer from neuromuscular diseases, such as amyotrophic lateral sclerosis, locked-in-syndrome (LIS), and spinal cord injury, to communicate with the external environment [4], [5]. EEG-based HCIs are also termed brain-computer interfaces (BCIs). The commonly used EEG patterns in BCIs include P300 potentials [6], [7], steady-state visually evoked potentials (SSVEPs) [8], and mu/beta rhythms related to motor imageries (MIs) [9], [10]. EOG can also be used to infer a user’s intentions based on his/her eye movements (blinking, fixating, and looking up/down/left/right). EOG-based HCIs allow users to select various commands by monitoring their eye movements [11], [12], [13], [14], [15]. These EEG- or EOG- based HCIs can work in a synchronous or asynchronous manner. A synchronous HCI assumes that the user is always in the control state and is instructed to issue each command according to a time cue provided by the system. An asynchronous (self-paced) HCI allows the user to freely switch between control and idle states and issue a command anytime [16], [17].
An EEG/EOG-based switch is a typical asynchronous HCI, in which the control and idle states need to be distinguished based on ongoing EEG/EOG signals [18]. The switch can be connected to a synchronous HCI system to control its activation or deactivation [19] or can be directly used to turn an external device on or off, e.g., an electric light, a TV, or an electrical prosthesis [18], [20]. For instance, for an HCI-controlled wheelchair, it is important to have a start/stop command, which must be elicited as promptly and accurately as possible. Such a start/stop command can be implemented by an asynchronous HCI, e.g., a switch. Similarly, as in other types of asynchronous HCIs, the most important and challenging issue for a switch is to rapidly and accurately distinguish the control and idle states. A strict discrimination criterion may result in a long response time (RT) in the control state, and conversely, a loose criterion may cause a high false positive rate (FPR) in the idle state.
Several studies have recently proposed switch designs based on EEG (brain switch) [21], [22], [23], [24], [25], [26]. For example, Mason et al. proposed a low-frequency asynchronous switch design (LF-ASD) based on a feature set related to MI in the 1–4 Hz frequency range [21]. Pfurtscheller et al. used the post-imagery beta rhythm associated with foot MI to develop a brain switch [22]. In our previous studies, a pseudo-key-based brain switch based on SSVEP [23], a hybrid brain switch combining P300 and SSVEP [24], and a P300-based threshold-free brain switch [25] were proposed. The authors of [26] utilized a P300-based or MI-based brain switch to produce a start/stop command for controlling a wheelchair. However, it is not easy to determine a satisfactory distinguishing criterion because EEG signals are highly variable and usually suffer from a poor signal-to-noise ratio (SNR) [27], [28]. Therefore, most of these EEG-based brain switches are characterized by long RTs in the control state and/or high FPRs in the idle state.
EOG is easier to detect than EEG because EOG is characterized by more consistent signal patterns. Furthermore, eye movements such as blinking are easy to perform and do not cause much discomfort to users [1]. Several EOG-based HCIs have been developed based on double blinks [12], [13], [29], triple blinks [12], [29], or winking (blinking monocularly) [30]. For example, Usakli et al. developed a virtual keyboard using EOG signals, in which a double blink was used as a selection method [13]. Norris et al. developed an eye-controlled mouse designed for the severely disabled, in which double blinks and triple blinks were utilized as a single and double mouse “click”, respectively [29]. Similarly, double blinks and triple blinks were used to control the “stop” and “go ahead” of a humanoid robot NAO (Aldebaran robotics, Inc.) in [12], respectively. In addition, winking (blinking monocularly) was also used as a trigger switch of the BCI system in [30]. Single blinks have been rarely used to develop switches, because people often make unintended/spontaneous single blinks [12]. The following challenges, which deteriorate the performance of existing EOG-based HCIs/switches, still exist: (i) It is not easy to quickly and accurately discriminate intended and unintended eye movements; and (ii) eye movements such as triple blinks or winking are not easy to perform by some users. For instance, as the authors of [30] noted, it is difficult for some subjects to blink twice or three times quickly, and some subjects blink three times when they intend to blink twice.
In this study, we propose a novel EOG-based asynchronous switch by integrating a visual trigger mechanism, which is similar to the P300 paradigm, to direct the user’s blinks and to provide accurate time for blinks detection. The hardware used to collect the vertical EOG signal includes three wet electrodes (one reference, one ground, and one signal electrode) and an acquisition device. The graphical user interface (GUI) consists of a switch button, which randomly flashes once in every period of 1.2 s. The flashing button is used to provide time stamps for the user to perform the intended blinks. Specifically, when the user seeks to issue an on/off command, he/she needs to blink synchronously with the flashes of the switch button, whereas the single-channel vertical EOG signal is collected and fed into a waveform detection procedure. Once an eye blink corresponding to the flash of a button is detected, an on/off command is issued. Thus, the intended and unintended blinks can be discriminated by the detection algorithm primarily based on the synchrony between the blink and the switch button’s flash. This EOG-based switch was further used in an intelligent wheelchair, which was developed in our previous study [31], to issue a start/stop command. Ten healthy subjects participated in three online experiments, and the experimental results demonstrated the effectiveness of the proposed system.
The remaining sections of this paper are organized as follows. The methodology, including system paradigm, detection algorithm, and system calibration, is presented in Section II. Section III briefly introduces the intelligent wheelchair system combining BCI and automated navigation techniques. The experiments and results are presented in Section IV. Further discussions of the method and results are provided in Section V, and Section VI concludes the study.
Section snippets
System paradigm
As shown in Fig. 1, the vertical EOG signal was measured using a monopolar electrode, denoted “A” and was referenced to the right mastoid, denoted “REF”. The ground electrode, “GND”, was positioned on the forehead [15]. The impedances of the electrodes were maintained below 5 kΩ. The single-channel EOG signal was acquired and amplified using a NuAmps device (Compumedics, Neuroscan, Inc., Abbotsford, Australia) and was sampled at 250 Hz.
Fig. 2 shows the graphical user interface (GUI) of the
An intelligent wheelchair system with an EOG-based switch
As an application, the proposed EOG switch was applied to an intelligent wheelchair for issuing a start/stop command. The intelligent wheelchair, which was proposed by Zhang et al. in [31], combined a BCI system based on MI + P300 and an autonomous navigation system. The working process of the intelligent wheelchair could be divided into the following four steps: (i) an obstacle map is constructed, and several candidate destinations (e.g., 25) are provided by an environmental perception module
Experiments
To evaluate the performance of the proposed EOG switch, ten healthy subjects (one female and nine males) from the local research unit, aged from 23 to 30, participated in three online experiments. All subjects had normal or corrected-to-normal vision. Each subject was instructed to read and complete an informed consent form before performing the experiments. For each subject, a 2-min training session was first performed to determine the three thresholds, as described in Section II-D: Calibration
Conclusion
In summary, this paper has presented a novel single blink-based EOG switch, which issues on/off commands depending on whether the user’s single blinks are performed in synchrony with the flashes of a switch button. The intended and unintended blinks can be easily discriminated primarily based on the synchrony between the blink and the flash of the switch button. Only the intended blinks trigger switch commands. As only one EOG channel is used, the proposed system is practically feasible in many
Acknowledgment
This work was supported in part by the National Key R&D Program of China under grant 2017YFB1002505, in part by the National Natural Science Foundation of China under Grant 61633010 and Grant 9142030002, and in part by Guangdong Natural Science Foundation under Grant 2014A030312005.
Yuanqing Li (F-2017) was born in Hunan, China, in 1966. He received the B.S. degree in applied mathematics from Wuhan University, Wuhan, China, in 1988, the M.S. degree in applied mathematics from South China Normal University, Guangzhou, China, in 1994, and the Ph.D. degree in control theory and applications from the South China University of Technology, Guangzhou, in 1997. Since 1997, he has been with the South China University of Technology, where he became a Full Professor in 2004. From
References (42)
- et al.
Brain-computer interfaces for communication and control
Electroenceph. Clin. Neurophysiol.
(2002) - et al.
The contingent negative variation and the late positive wave of the average evoked potential
Electroenceph. Clin. Neurophysiol.
(1970) - et al.
Mu rhythm (de) synchronization and EEG single-trial classification of different motor imagery tasks
Neuroimage
(2006) - et al.
Could the beta rebound in the EEG be suitable to realize a “brain switch”?
Clin. Neurophysiol.
(2009) - et al.
The PaTa criterion and rejecting the abnormal value
J. Zhengzhou Univ. Technol.
(1997) - et al.
Performance reliability estimation method based on adaptive failure threshold
Mech. Syst. Signal Process.
(2013) - et al.
Development of NASA-TLX (task load index): results of empirical and theoretical research
Adv. Psychol.
(1988) - et al.
Diurnal variation in spontaneous eye-blink rate
Psychiatry Res.
(2000) - et al.
Efficient implementation and design of a new single-channel electrooculography-based human-machine interface system
IEEE Trans. Circuits Syst. II: Express Briefs
(2015) Über das elektrenkephalogramm des menschen
Arch. Psychiat. Nervenkr.
(1933)
System for assisted mobility using eye movements based on electrooculography
IEEE Trans. Neural Syst. Rehabil. Eng.
Extra-Ordinary Human Computer Interaction
An online semi-supervised brain-computer interface
IEEE Trans. Biomed. Eng.
Brain-computer interfaces based on the steady-state visual-evoked response
IEEE Trans. Rehabilit. Eng.
An EEG-based BCI system for 2-d cursor control by combining mu/beta rhythm and p300 potential
IEEE Trans. Biomed. Eng.
Eye movement analysis for activity recognition using electrooculography
IEEE Trans. Pattern Anal. Mach. Intell.
A novel EOG/EEG hybrid human–machine interface adopting eye movements and ERPs: application to robot control
IEEE Trans. Biomed. Eng.
Design of a novel efficient human–computer interface: an electrooculogram based virtual keyboard
IEEE Trans. Instrum. Meas.
A hybrid platform based on EOG and EEG signals to restore communication for patients afflicted with progressive motor neuron diseases
Conference Proceedings of the IEEE Engineering in Medicine and Biology Society
Controlling a human–computer interface system with a novel classification method that uses electrooculography signals
IEEE Trans. Biomed. Eng.
Asynchronous p300-based brain–computer interfaces: a computational approach with statistical models
IEEE Trans. Biomed. Eng.
Cited by (32)
Research on a spatial–temporal characterisation of blink-triggered eye control interactions
2024, Advanced Engineering InformaticsOnline interaction method of mobile robot based on single-channel EEG signal and end-to-end CNN with residual block model
2022, Advanced Engineering InformaticsCitation Excerpt :Table 3 presents the comparison of this study with state-of-the-art BCI-based control researches. Comparing with the existing studies [4,16,19,20,22,23,32,37,45–49,52,53], the main advantages of this paper are as follows. The detection of LEB, REB, CEB, and GT tasks is realized by using single channel dry electrode EEG signals.
Multimodal analysis of electroencephalographic and electrooculographic signals
2021, Computers in Biology and MedicineCitation Excerpt :Based on various tasks (resting, writing, and reading), the performance of the intended model is measured with two traditional models to demonstrate the improved accomplishment of the implemented model. With the assistance of an EOG-based switch architecture, Li et al. [12] have implemented a visual trigger mechanism to direct user blinks and enhance their detection. The waveform recognition algorithm is applied to the continuous EOG signal, which distinguishes between expected and unintentional blinks.
RVEB—An algorithm for recognizing voluntary eye blinks based on the signal recorded from prefrontal EEG channels
2020, Biomedical Signal Processing and ControlCitation Excerpt :A thresholding operation was then applied to find the parts of the signal of the amplitude exceeding +/−4 V and width exceeding 400 ms. The authors reported an accuracy of 85 % averaged over 20 subjects. Li et al. [14] used a single switching command, derived from EEG signal, to start/stop a wheelchair. A novel feature of the solution proposed by these authors was to use a graphical user interface (GUI) with a button that flashed every 1.3 s.
Design and development of an EOG-based simplified Chinese eye-writing system
2020, Biomedical Signal Processing and Control
Yuanqing Li (F-2017) was born in Hunan, China, in 1966. He received the B.S. degree in applied mathematics from Wuhan University, Wuhan, China, in 1988, the M.S. degree in applied mathematics from South China Normal University, Guangzhou, China, in 1994, and the Ph.D. degree in control theory and applications from the South China University of Technology, Guangzhou, in 1997. Since 1997, he has been with the South China University of Technology, where he became a Full Professor in 2004. From 2002 to 2004, he was with the Laboratory for Advanced Brain Signal Processing, RIKEN Brain Science Institute, Saitama, Japan, as a Researcher. From 2004 to 2008, he was with the Laboratory for Neural Signal Processing, Institute for Infocomm Research, Singapore, as a Research Scientist. He is the author or coauthor of more than 80 scientific papers in journals and conference proceedings. He is an IEEE Fellow. His research interests include blind signal processing, sparse representation, machine learning, braincomputer interface, EEG, and fMRI data analysis.
Shenghong He received the B.S. degree in automation from the South China University of Technology, Guangzhou, China, in 2011. He is currently working toward the Ph.D. degree in pattern recognition and intelligent systems at South China University of Technology. His research interests include pattern recognition and noninvasive brain–computer interfaces.
Qiyun Huang received the B.S. degree in automation from Beihang University, Beijing, China, in 2013, and the M.Sc. degree in electrical engineering from University of Southern California, Los Angeles, USA, in 2015. Currently, he is a Ph.D. candidate in pattern recognition and intelligence systems at South China University of Technology, Guangzhou, China. His research interests include noninvasive brain–computer interface and pattern recognition.
Zhenghui Gu received the Ph.D. degree from Nanyang Technological University in 2003. From 2002 to 2008, she was with Institute for Infocomm Research, Singapore. She joined the College of Automation Science and Engineering, South China University of Technology, in 2009 as an associate professor. She was promoted to be a full professor in 2015. Her research interests include the fields of signal processing and pattern recognition.
Zhu Liang Yu received his BSEE in 1995 and MSEE in 1998, both in electronic engineering from the Nanjing University of Aeronautics and Astronautics, China. He received his Ph.D. in 2006 from Nanyang Technological University, Singapore. He joined Center for Signal Processing, Nanyang Technological University from 2000 as a research engineer, then as a Group Leader from 2001. In 2008, he joined the College of Automation Science and Engineering, South China University of Technology and was promoted to be a full professor in 2011. His research interests include signal processing, pattern recognition, machine learning and their applications in communications, biomedical engineering, etc.
- 1
These authors contributed equally to the manuscript.