Unsupervised automatic online spike sorting using reward-based online clustering

Highlights

• The novel method is Reward Based Online Clustering.

• A modified method for feature extraction called Aligned First Derivative with just two features is proposed.

• The graphical results for experimental data obtained from rat brain approve the efficiency of the proposed method.

• In comparison to the other similar works the proposed method has more accurate clustering results.

Abstract

Brain-machine interfaces (BMIs) can enable paralyzed people to regain mobility. In these interfaces, some different type of signals can be obtained from the brain, one of which is the action potential waveform (spike). In the case of using spikes, sorting the recorded signals and isolating the effects of the individual neurons can lead to a greater efficiency. Also, because of the nature of BMIs, real-time spike sorting is necessary. In many spike sorting approaches, the main outline consists of the following steps: spike detection, feature extraction, and clustering. In this study, a novel method for clustering is presented. This method is referred to as Reward-Based Online Clustering (RBOC) which is formed based on the reinforcement learning algorithm. The significant property of this proposed technique is its capability for real-time implementation that is required by BMIs. This method can automatically detect the clusters while there is no knowledge about the number of clusters. The performance of the proposed method is demonstrated through both simulation and experimental study. Evaluation with artificially simulated (ground truth) data shows that, on average, the accuracy of categorizing the spikes from the same origins is above 94 percent. Moreover, implementation of the method on the experimental data obtained from the rat brain represents convincing sorting results. It is noteworthy to say that, in most cases, this new method outperforms the results of similar previous works.

Introduction

One of the basic approaches for understanding brain mechanisms is monitoring the electrical activity of the neurons [1] which helps us to decode the brain information which is vital for BMIs. These neural electrical signals are usually recorded with one or more electrodes, but each electrode records a superposition of activities of several adjacent and farther neurons. The adjacent neurons to the electrode shape a spike train and the farther neurons contribute to the background noise [1]. Hence, spike sorting is required to distinguish each part of the recorded signals and obtain information from the brain [2]. In addition, it has been shown that even a basic spike sorting is beneficial for BMI signal decoding [3].

There are wide variety of spike sorting approaches on neuronal pattern recognition [4]. Most spike sorting algorithms comprise three major steps: spike detection, feature extraction, and clustering [5].

In spike detection, the common method is applying a simple threshold to extract the actual spike waveforms from the raw data [4]. Besides, more complicated techniques, which are based on the wavelet method, such as wavelet transform (WT) [6] or wavelet packets decompositions (WPD) [7] and the energy signal [8,9] for reducing harmful effects on the spike shape and spike detecting, are introduced. In [10] a sparse coding and a compressive sensing method are introduced to solve the problem of overlapping spikes. Moreover, in order to simplify the spike detection step, some studies use mathematical transformation to obtain a new signal from the raw signals that identified spikes in a simple manner [11,12].

Feature extraction is mapping from the original data space to another space, and features are properties of a cluster that specify it from the other clusters. Besides, feature extraction can map the data in a space with lower dimension and as a result, reduce the computational complexity. The other desirable property of feature extraction is that it makes a cluster more distinct, so, it facilitates the clustering process [13]. The most common method for feature extraction and order reduction, which is used in spike sorting literature, is Principal Component Analysis (PCA) [1] which is mostly used in offline clustering methods. Moreover, Wavelet coefficients are proper features to discriminate clusters, but they are complex [6,7]. The other features are the first and second extrema (FSDE) method that are simple shape-based features [13]. There are other features like Diffusion Map features that use a graph of the data for feature extraction [14], and Locality Preserving Projection (LPP) that constructs a graph incorporating neighborhood information of the dataset [15], for offline applications. In contrast, the FSDE features that can be applied for online applications. Recently, a feature extraction approach is introduced that uses shape, phase, and distribution together [16]. It seems that the shape-based feature extractions are more appropriate for online applications rather than other mapping-based methods.

The main part of most spike sorting methods is the clustering technique. Hence, different clustering approaches are presented in the literatures, such as k-means [7,13,14,17], expectation maximization [18,19], superparamagnetic clustering method [6], landmark-based spectral clustering [15], hierarchical adaptive means (HAM) clustering [20], neural networks [8], support vector machines (SVMs) [21]. However, none of the mentioned approaches successfully provided generally optimal results in sorting the spikes. In addition, because of the nature of BMIs, real-time spike sorting is necessary and none of above-mentioned clustering methods satisfy this requirement. Hence, in this study, a new method for online clustering is proposed. This method is Reward-Based Online Clustering (RBOC) which is formed based on the reinforcement learning algorithm. The significant property of this proposed technique is the capability of real-time implementation which is required by BMIs. Besides, this method can automatically detect the clusters while there is no knowledge about the number of clusters.

Reinforcement learning algorithms such as Q-learning have been built based on interaction with the environment and receiving a reward or punishment [22]. Reward or punishment signal can be a physical or real signal, such as a banana given to a monkey for accomplishing a task [23]. It also can be a purely mathematical definition for example distance with the target for an agent. Such algorithms are acting real-time, i.e., when the algorithm starts, information or experience completes by receiving the reward or punishment signal. Because the experience can take place without a supervisor, the algorithm can be done unsupervised. In this case, only the reward or punishment signal definition is required for the algorithm. Besides, this algorithm can be used as an unsupervised automatic real-time clustering method.

Each experience accompanies with a cost and takes plenty of time. Therefore, the concept of planning helps reinforcement learning algorithm to be faster and cheaper. In the planning algorithm, along with the experience, a model of the system is built. The simulated model helps to gain virtual experiences quickly without real interaction with the environment. Then the algorithm would be faster [24].

In this study, the procedure consists of three-parts, detection using simple threshold, feature extraction with Aligned First Derivative (AFD) method, and the new main part which is online unsupervised clustering method with Reward-Based Online Clustering (RBOC) approach.

The evaluation is performed with a set of simulated datasets. Each dataset simulates one trial of recording the brain signal using a single electrode. These sets are classified into four categories based on the degree of complexity regarding the similarity of the spikes in the spike train. Each smaller dataset has four different noise levels. After evaluation, the proposed method is applied to the experimental data obtained from the rat brain using a single electrode.