Online learning using projections onto shrinkage closed balls for adaptive brain-computer interface
Introduction
Online learning updates the model on the fly using only single or a chunk of new samples [1], [2]. Compared to traditional batch learning that needs all training samples to be prepared beforehand, online learning exhibits three major advantages. First, it allows data scale to grow infinitely, and thus is well-suited for endless or lifelong learning [3]. Second, online learning yields an up-to-date model by adapting to data dynamics, making it capable of handling non-stationary issues, such as the concept shift [4]. Third, online learning preserves history information, thus avoiding repetitive batch retraining from scratch and significantly reducing the computational effort. With these advantages, online learning is nowadays widely explored in various fields, such as big-data analysis [3], large-scaled medical image-based diagnosis [5], video stream data analysis [6], [7], non-stationary physiological signal recognition [8], wearable health-monitoring apparatus [9], and so on. In the present study, we will focus on online learning for brain-computer interfaces (BCIs).
By decoding the information of electroencephalography (EEG) signals, BCIs provide a feasible way for implementing direct brain-controlled devices [10], [11], or intelligent devices that perceive the user’s mental states [12]. Wearable/portable BCIs, holding the promise of bringing the fruitful outcome of decades of laboratory BCI studies to daily lives of end users, recently draw the attention of researchers. Several such experiments/prototypes have been reported, e.g., mobile P300 spellers [13], [14], wearable navigation systems [15], [16], assistive robot devices [17], head-mounted device (HMD)-based augmented reality (AR) systems [16], [18], wearable mental-load evaluation systems [19], [20], as well as convenient EEG sensors [21], [22], [23]. However, due to the resource constraint, the well-performed batch methods, such as support vector machine (SVM) [24], step-wise linear discriminant analysis (SWLDA) [25], and more recently the Riemannian [26] and tensor-based [27] approaches, etc., normally need the classifier to be trained on another professional computer before it is applied to the wearable/portable device. A challenge is how to update the model to meet the EEG changes, which may come from either internal (mental) or external (environmental) variations, and cause a drop of performance during the long-term use of a BCI. Although an out-of-device batch retraining could be a solution, an on-device updating or training would be more convenient and desirable. Online learning, benefiting from its adaptability and high-efficient computation, naturally fits for this purpose. Several methods have been evaluated, such as the online SVM [28], [29], the passive aggressive (PA) algorithm [30], [31], etc. However, online learning still needs to be examined in the field of the BCI, especially for the event-related potential (ERP)-based BCI, or traditionally called the P300 speller (see F. Lotte’s recent review on BCI classification algorithms [32]).
The sources causing the EEG changes may include: 1) EEG non-stationarity, 2) spatial-domain variation, and 3) poor recordings. EEG is believed to be short-term stationary, and thus the BCI performance can keep stable for hours or even for days. However, EEG non-stationarity cannot be overlooked from a long-term view. It may affect morphologies and latencies of the underlying ERP components, and deteriorate the BCI performance. EEG non-stationarity may be linked to a variety of factors, such as learning effect, habituation, age, etc., but is still not fully understood. Transfer learning has been used to handle the EEG changes between sessions or even between subjects [28], [29], [33], [34]. The basic concept under transfer learning is to use data recorded in one task to boost performance in another task [35]. For a BCI, the main role of transfer learning is to reduce the calibration effort, by using data from previous sessions or other subjects. Similarly, semi/un-supervised or self-calibration methods have also been employed for this purpose [36], [37], [38]. The calibration of a BCI is normally tedious and time-consuming for the end user, and how to reduce the calibration effort becomes an important issue in the BCI research. However, it is beyond the scope of the present study. Compared to EEG non-stationarity, due to the location-specific assumption of ERPs, a more severe sudden drop of performance may occur with spatial-domain variations, which are often caused by a shift or switch of headset every time wearing the device. To ease switching of headsets, D. Wu et al. [39] proposed an active learning method, and showed its effectiveness for reducing the recalibration effort from a new headset. Spatial-domain variations may also come from spatial-distribution changes of cortical activities. To handle this kind of variations, spatial filtering methods, such as xDAWN [40] and its adaptive form, axDAWN [30], etc., which aim to find the optimal spatial combination in multi-electrode EEG recordings, have been applied. As the third source, poor recordings, mainly arising from a sensor quality/impedance change, external noises, artifacts, etc., may worsen the signal quality. This kind of EEG changes can be compensated by the dynamic stopping technique [29], [41], whose basic idea is to adaptively adjust the amount of data required for judgment, by decreasing the amount of data at good signal-to-noise ratios (SNRs) and increasing it at bad SNRs, to maintain a high-enough BCI throughput at different signal quality levels. It should be noted that, because dynamic stopping is based on the assumption that only the relative noise level changes, it is often used in combination with other methods, such as transfer learning.
In the above, we have briefly summarized the main reasons for EEG changes and the coping strategies to date in the field of the BCI. Though performing well, some of them, such as transfer learning [28], [29], [33], [34] and active learning [39], are computationally demanding, whereas others, such as spatial filtering [30], [40] and dynamic stopping [29], [41], may work in conjunction with online learning. Meanwhile, the label retrieving strategies taken by semi/un-supervised methods [37], [38] could also be used by online learning, in cases when true labels are unavailable.
Although using sliding buffers, it is possible to transfer batch methods, such as SVM [28], as well as their incremental forms [29], to the online setting, several drawbacks may be encountered. One possible drawback is a drop of performance arising from reduced training samples. As we know, batch methods, such as SVM, generally need large-scaled samples to get high generalization performance. Another possible drawback is, due to a lack of adaptability, data selections may be required, indicating an increased computational effort. Therefore, a genuine online algorithm would be preferred.
We investigated an adaptive projected subgradient method (APSM) [42] for online learning of the BCI. APSM is derived based on the set theoretic estimation formulation and the projections onto convex sets theory in reproducing kernel Hilbert space (RKHS), and provides a unifying framework for both adaptive classification and regression tasks. Concurrent projections of APSM also make it possible to boost calculating efficiency through parallel computation. In the present study, we extended the general APSM to a shrinkage form. The convergence of the proposed method was proved. It was also shown that several new valuable properties, i.e., a stable and controllable fading effect, the easiness of tuning, the least norm solution, etc., were obtained. The performance of the proposed method was evaluated, and compared with those of the general APSM, the incremental support vector machine (ISVM) [29], and the PA algorithm [30], [31], through an ERP-based BCI experiment. Beyond the BCI, the proposed method potentially can also be used in other related fields that suffer from the EEG changes, such as for tackling the Template Aging issue in EEG-based biometrics [43].
Section snippets
The general APSM
Given a set with yi the label of an observed sample xi, and a RKHS defined by a positive definite kernel APSM tries to find a classifier, where through the following sequence [42], [44]where PC( · ) is the projection operator associated with a closed convex set C; with a closed ball with some predefined radius δ > 0; denotes
Results
As shown in Fig. 3, compared to the CF speller, improved ERP magnitudes can be obviously observed for the VF speller, mainly manifested by the enhanced positive peaks at the posterior area (O1/O2) and negative peaks at the frontal area (Fz) around 300 ms, and the enhanced negative peaks at posterior area (O1/O2) around 170 ms.
Figs. 4 and 5 show the online classification performance of each methods, when a 80ms latency shift was introduced to CF and VF, respectively. It can be seen that
Discussion
Our results show that shrinkage APSM outperforms the general APSM and PA in both the recovery duration and the recovered accuracy, indicating an improved adaptability for shrinkage APSM. Recovery duration measures the time period the model needs to recover to an acceptable performance, e.g. to a CWA above 80%, from a data shift, whereas recovered accuracy measures how well the classifier performs after training for a certain period of time following a data shift. From our results, it can be
Conclusion
A BCI equipped with online learning is able to update itself in time to cope with possible EEG changes. We investigated an online learning method called the APSM, and derived its shrinkage form by using projections onto shrinkage balls whose radii vary with data, which was then demonstrated to yield a stable fading effect that helps to improve the online classification performance of the BCI. Several useful properties of shrinkage APSM were given and proved. It was also shown that, as the
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China (61772508, U1713213, 61906183), Shenzhen Technology Project (JCYJ20170413152535587, JCYJ20180507182610734), Key Research and Development Program of Guangdong Province (2019B090915001), CAS Key Technology Talent Program.
Zheng Ma received the B.E. degree in electronic engineering from Dalian University of Technology, Dalian, China, in 2006, and the Ph.D. degree in biomedical engineering from Dalian University of Technology, Dalian, China, in 2016. Currently, he is a postdoctoral fellow in the Laboratory for Human Machine Control, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China. His research interests include human-machine interaction, brain-computer interface, and
References (48)
- et al.
One-pass online learning: a local approach
Pattern Recognit.
(2016) - et al.
Chunk incremental learning for cost-sensitive hinge loss support vector machine
Pattern Recognit.
(2018) - et al.
Incremental on-line learning: a review and comparison of state of the art algorithms
Neurocomputing
(2018) - et al.
Variational inference based Bayes online classifiers with concept drift adaptation
Pattern Recognit.
(2018) - et al.
Heterogeneous data analysis: online learning for medical-image-based diagnosis
Pattern Recognit.
(2017) - et al.
Reinforcement online learning for emotion prediction by using physiological signals
Pattern Recognit. Lett.
(2018) - et al.
Online anomaly detection for long-term ECG monitoring using wearable devices
Pattern Recognit.
(2019) - et al.
Towards an affordable brain computer interface for the assessment of programmers’ mental workload
Int. J. Hum.-Comput. Stud.
(2018) - et al.
Online localization and prediction of actions and interactions
IEEE Trans. Pattern Anal. Mach. Intell.
(2019) - et al.
First-person activity forecasting from video with online inverse reinforcement learning
IEEE Trans. Pattern Anal. Mach.Intell.
(2018)
Brain-computer interfaces for communication and rehabilitation
Nat. Rev. Neurol.
Visual and auditory brain-computer interfaces
IEEE Trans. Biomed. Eng.
Passive BCI in operational environments: insights, recent advances, and future trends
IEEE Trans. Biomed. Eng.
Spelling with a small mobile brain-computer interface in a moving wheelchair
IEEE Trans. Neural Syst. Rehab. Eng.
A transplantation of subject-independent model in cross-platform BCI
Int. J. Mach. Learn. Cybern.
Towards BCI-actuated smart wheelchair system
Biomed. Eng. Online
A wearable SSVEP-based BCI system for quadcopter control using head-mounted device
IEEE Access
EEG-based BCI control schemes for lower-limb assistive-robots
Front. Hum. Neurosci.
Towards BCI-based interfaces for augmented reality: feasibility, design and evaluation
IEEE Trans. Visual. Comput. Graph.
Anxiety level detection using BCI of miner’s smart helmet
Mob. Netw. Appl.
Unobtrusive ambulatory EEG using a smartphone and flexible printed electrodes around the ear
Scient. Rep.
In-ear EEG from viscoelastic generic earpieces: robust and unobtrusive 24/7 monitoring
IEEE Sens. J.
An inflatable and wearable wireless system for making 32-channel electroencephalogram measurements
IEEE Trans. Neural Syst. Rehab. Eng.
BCI competition III: dataset II- ensemble of SVMs for BCI P300 speller
IEEE Trans. Biomed. Eng.
Cited by (4)
Interface, interaction, and intelligence in generalized brain–computer interfaces
2021, Trends in Cognitive SciencesCitation Excerpt :When applying electrical stimulation, additional problems such as the lifetime of implanted electrodes, artifacts produced by electrical stimulation, and electrochemical safety of electrode–tissue interface arise and remain to be solved [27]. A robust implementation of a closed-loop BCI system depends on the co-adaptation between the brain and the decoder [39–41]. On the one hand, the brain should adapt to the changes in the external environment, and constantly optimize the execution of tasks; on the other hand, the decoder and the external actuators should also learn to adapt to the changes in neural activities and correctly identify the user’s intention.
Online kernel classification with adjustable bandwidth using control-based learning approach
2020, Pattern RecognitionCitation Excerpt :Online learning plays a crucial role in machine learning community due to its extensively applications on realistic modeling problems such as time series analysis [1], image processing [2], objective tracking [3], financial quantitative transaction [4] and pattern recognition [5].
A Review of Adaptive Brain-Computer Interface Research
2023, Dianzi Yu Xinxi Xuebao/Journal of Electronics and Information TechnologyA Calibration-free Approach to Implementing P300-based Brain–computer Interface
2022, Cognitive Computation
Zheng Ma received the B.E. degree in electronic engineering from Dalian University of Technology, Dalian, China, in 2006, and the Ph.D. degree in biomedical engineering from Dalian University of Technology, Dalian, China, in 2016. Currently, he is a postdoctoral fellow in the Laboratory for Human Machine Control, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China. His research interests include human-machine interaction, brain-computer interface, and machine learning.
Jun Cheng received the B.E. and M.E. degrees from the University of Science and Technology of China, Hefei, China, in 1999 and 2002, respectively, and the Ph.D. degree from the Chinese University of Hong Kong, Hong Kong, in 2006. He is currently with the Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China, as a Professor, and the Director of the Laboratory for Human Machine Control. His current research interests include computer vision, robotics, and machine intelligence and control.
Dapeng Tao received a B.E. degree from Northwestern Polytechnical University and a Ph.D. degree from South China University of Technology, respectively. He is currently a professor with School of Information Science and Engineering, Yunnan University, Kunming, China. He has authored and co-authored more than 60 scientific articles. He has served more than 10 international journals including IEEE TNNLS, IEEE TIP, IEEE TCYB, IEEE TMM, IEEE CSVT, IEEE TBME, and Information Sciences. Over the past years, his research interests include machine learning, computer vision and robotics.