Abstract
BCI (brain computer interface) is particularly important for HCI. Some of recent results concerning BCI made a great contribution to the development of the HCI research area. In this paper we define “good-trained brain states”, and then propose a method for discriminating good-trained brain states from other states. We believe that repetitious training might be effective to human brains. Human brain reactions can be quantified by ERPs (event related potentials). We analyze the data of ERPs reflecting the brain reactions, and then discuss the effect of repetitious training to the brain states.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
In order to achieve a satisfactory stage of the effective BCI, we consider a method of how to discriminate good brain states from other states. Some skills can be improved by repetitious training. Using a BCI system we can observe some changes in brain states by the repetitious training. In our experiments, we use such training for simple calculation tasks by subjects. We recognize some noticeable changes [5, 6] in their ERPs [8, 12–14] during the repetitious tasks [4]. At a certain stage of each experiment, we notice that their ERPs reach a stable state within minor fluctuation [1, 5]. We call such a stable state a good-trained brain state. In our experiments, the good-trained brain state shows that the skill for the simple calculation reaches the proficient level as the result of the repetitious training. That is, the good-trained brain state reflects the effect of the repetitious training. By analyzing the ERPs obtained from our experiments, we propose a method for discriminating good-trained brain states from other states [6].
According to the review of BCI [9], BCI can be considered to be an artificial intelligence system recognizing a certain set of patterns in brain signals through the following five consecutive stages: signal acquisition, preprocessing or signal enhancement, feature extraction, classification, and the control interface. In this paper we focus our attention on the first four stages of the BCI.
2 Methods
2.1 An Experimental Method
Each of the following experiments is repeatedly carried out (eight times per each task on the average).
The subjects are 10 men and 2 women from 20 to 23 years old. All of them are right-handed. Each of the subjects is identified by a letter from a to l. The experiments were carried out in the laboratory of the first author at Hakuoh University. We use two kinds of stimuli. One is a division question and the other is a circle, as shown in Fig. 1. Each stimulus is displayed in a CRT (cathode ray tube) of 19 inches with 80 × 240 pixels.
A subject starts his/her calculation work when a division question is displayed. He/she inputs his/her answer to the question when a circle is displayed.
We call the calculation work, from when a division question is displayed until when its answer is given by a subject, a task. A sequence of sets of stimuli is displayed in the CRT placed in front of the subject. Each sequence consists of 100 tasks. A circle displayed after each division question is to inform the subject the timing for answering (as shown in Fig. 1). The circle is displayed for 1 s. The time interval between two consecutive stimuli is randomly chosen within the range from 800 [ms] to 1200 [ms]. EEGs as a response to the stimuli, a set of a division question and a circle followed by the question, are recorded in real time (strictly speaking, it requires 1 s to record the EEGs). Consequently about 10 min are required to record the data for one experiment (i.e., 100 division questions together with answers). The single polar and four channels of the “international 10–20 method” are used to measure the EEGs. The positions of the measurement are at C3, C4, Cz, and Pz. The base is A1 that is connected to A2. The sampling frequency for the A/D converter is 1 kHz.
2.2 An Analytical Method
The recorded EEGs shown in Fig. 4 are filtered by an adaptive filter [10, 11], and the EEGs are normalized by taking the average of their waveforms and using the standard deviation of the data (see Fig. 5). Then we obtain the ERPs of 100 repetitious tasks by using the normalized EEGs, the AM (averaging method), the DSAM (data selecting and averaging method) [3], and the m-DSAM (multi-DSAM) [6].
2.2.1 Data Selecting and Averaging Method (DSAM)
The data of EEGs are sieved and normalized by using a threshold value, and then the selected and normalized data are averaged. In this way ERPs are derived from EEGs. This method is called the DSAM (data selecting and averaging method). The outline of the data process by the DSAM is as follows:
Let xi(t) be a filtered and normalized data, where i is a data index and t is the latency (the time delay from the time when a division question is displayed). Let D be a data set of xi(t).
-
Step 1: For each i, xi(t) is translated into the binary sequence bi(t) by threshold L (in this paper L = 0.5, which is determined by signal-noise ratios), i.e.
$$ b_{i} (t) = \left\{ {\begin{array}{*{20}c} 1 & {if} & {x_{i} (t) > L} \\ 0 & {if} & {x_{i} (t) \le L} \\ \end{array} } \right.. $$ -
Step 2: The sum B(t) of all bi(t) is calculated.
-
Step 3: The maximum value MB of B(t) around the latency of a positive peak P is found. Let TP be the latency such that B(TP) = MB. In our analysis of the data obtained by DSAM, P = P3 is chosen.
-
Step 4: We find the subset DP3 of D such that
$$ D_{\text{P3}} = \left\{ {x_{i} (t)|x_{i} (T_{\text{P3}} ) > L,\quad x_{i} (t) \in D} \right\}. $$ -
Step 5: We calculate
\( ERP_{\text{DSAM}} (t) = \frac{1}{{n_{\text{P3}} }}\sum\limits_{{x_{i} (t) \in D_{\text{P3}} }} {x_{i} (t)} \), where nP3 is the number of elements in DP3.
2.2.2 Multi-data Selecting and Averaging Method (M-DSAM)
The m-DSAM is a method to find the potential of an ERP for each target (peak) by using the DSAM. For each potential obtained in this way, we estimates the ERP by a normal distribution. The outline of the data process by the m-DSAM is as follows:
We modify the process in Step 1 of Sect. 2.2.1 in order to detect negative peaks.
-
Step 1: For each i, xi(t) is translated into the binary sequence bi’(t) by threshold L’ (= −0.5), i.e.
$$ b_{i} '(t) = \left\{ {\begin{array}{*{20}c} 0 & {if} & {x_{i} (t) \ge L'} \\ 1 & {if} & {x_{i} (t) < L'} \\ \end{array} } \right.. $$ -
Step 2: We calculate the sum B(t) of all bi(t) and the sum B’(t) of all bi’(t).
-
Step 3: For each positive peak P, we find the maximum value MBP of B(t). For each negative peak N, we find the maximum value MBN of B’(t), too. The interval between two peaks can be predicted from the past data.
-
Step 4: We find the subsets DP and DN of D such that
$$ D_{\text{P}} = \left\{ {x_{i} (t)|x_{i} (T_{\text{P}} ) > L,\,\,x_{i} (t) \in D} \right\},\quad D_{\text{N}} = \left\{ {x_{i} (t)|x_{i} (T_{\text{N}} ) < L',\,\,x_{i} (t) \in D} \right\}. $$ -
Step 5: We calculate
$$ ERP_{\text{P}} (t) = \frac{1}{{n_{\text{P}} }}\sum\limits_{{x_{i} (t) \in D_{\text{P}} }} {x_{i} (t)} ,\quad ERP_{\text{N}} (t) = \frac{1}{{n_{\text{N}} }}\sum\limits_{{x_{i} (t) \in D_{\text{N}} }} {x_{i} (t)} , $$where np and nN are the numbers of elements in Dp and DN, respectively.
-
Step 6: For each peak P or N, we estimate ERPP(t) or ERPN(t), respectively. These values are calculated as the values of ERPs at peaks P or N, by the following formulae (a normal distribution determined by a certain algorithm [2]):
$$ ERP_{\text{Q}} (t) \approx f_{\text{Q}} (t) = ( - 1)^{h} w_{\text{Q}} \frac{1}{{\sqrt {2\pi } s_{\text{Q}} }}\exp \left( { - \frac{{(t - T_{\text{Q}} )^{2} }}{{2s_{\text{Q}}^{2} }}} \right), $$(1)where Q is a P or an N, h(Q) is the function such that if Q = P then h = 0, otherwise h = 1, wQ is the amplitude of peak Q, sQ is the standard deviation, and TQ is the latency of Q.
-
Step 7: We calculate
\( ERP_{\text{m - DSAM}} (t) = \frac{1}{{\bar{n}}}\sum\limits_{\text{Q}} {n_{\text{Q}} f_{\text{Q}} (t)} \), where \( \bar{n} \) is the average of all \( n_{\text{Q}} \)’s.
2.2.3 The Definition of the Distance Between Potentials
In our experiment, positive peaks P1, P2 and P3, and negative peaks N1, N2 and N3 are closely related to the task. Let Q denote one of these peaks. Let j and k be experimental days. We let \( {\mathbf{v}}_{{j{\text{Q}}}} = \left( {w_{{j{\text{Q}}}} ,\,\,s_{{j{\text{Q}}}} ,\,\,T_{{j{\text{Q}}}} ,\,\,n_{{j{\text{Q}}}} } \right) \).
Then we can construct formulae (1) from this vector, too. We consider that vector vjQ represents the potential at the peak Q on the j-th experimental day. Let vjQ and vkQ be the vectors for experimental days j and k, respectively. We let
Then we can write vjQ = uj/kQ * vkQ, where the operation “*” is the Hadamard product. Next we define a distance between uj/kQ and uk/kQ of ERPs at peak Q on j-th experimental day and k-th experimental day as the Euclidean distance, denoted by d(uj/kQ, uk/kQ).
An ERP can be reconstructed by the m-DSAM and the linear combination of the normal distributions. Four parameters for each potential are determined by the m-DSAM: three of these parameters are shown in Fig. 2. Then we define a distance between vectors of these parameters with the coordinates determined by the m-DSAM. Using this distance, we discriminate the good-trained brain states from other brain states.
3 Results
3.1 The Ratios of Correct Answers
We calculate the ratios of correct answers (RCA) for all subjects and experimental days. Then we calculate the average of them and the standard deviations for each experimental day. These values are shown in Fig. 3, where the maximum value of the averaged RCA is 76.1 %. In Fig. 3, the horizontal line marked by tiny triangles on the line shows 95 % of 76.1 %. The area of the graph above the horizontal line can be considered to be the area showing that the subject reaches the proficient level of the calculation task.
3.2 The Definition of a Good-Trained Brain State
We analyze the RCAs by the cluster analysis regarding to the experimental days for each subject. As shown in Table 1, the RCAs can be classified into two groups. The RCAs used for the classification are different among the subjects. Comparing Fig. 3 and Table 1, we consider that this classification is reasonable. In Table 1, the brain states indicated by “g” are good-trained brain states. The entries with oblique lines in Table 1 indicate that we could not carry out the calculation for the cluster analysis due to the loss of the experimental data.
3.3 ERPs Calculated by the m-DSAM
In this section, we give a number of examples calculated by the m-DSAM. The graph in Fig. 4 shows the recorded data (EEGs), which were obtained from the 3rd data measurement of subject g at the 2nd experimental day. The data contains 50 Hz alternative current noise and other types of noise as well. The graph of xi(t) (i = 3, t = 1,…, 1000) in Fig. 5 shows the data after filtering and normalizing the data given in Fig. 4. Then we translate x3(t) into the binary sequences b3(t) and b3’(t). These translated data are shown in Fig. 6. Then we calculate bi(t), bi’(t), the summation B(t) of bi(t), and the summation B’(t) of bi’(t) for i = 1, …, 100. The summations B(t) and B’(t) are shown in Fig. 7. It was previously reported that the potentials P1, N1, …, P3, and N3 were noticed on the graph of the ERPs, and that the latencies of the ERPs were also estimated. Analyzing the experimental data, we group B(t) and B’(t) into several time intervals. For each interval of B(t) and B’(t), we find the maximum values MBP and MBN, respectively, and determine the latencies TP and TN such that B(TP) = MBP and B’(TN) = MBN.
The data shown in Fig. 4 are filtered and normalized
The binary sequences translated from the data shown in Fig. 5
For each peak of Q = P1, N1, …, P3, and N3, we determine the parameters as described at Step 6 in Sect. 2.2.2, and then we obtain Table 2. In Fig. 8, the dashed curves and the solid curve show fQ(t)’s and ERPm-DSAM(t), respectively.
3.4 Distances Between Two Potentials
Let ksb be the kth experimental day when a subject sb reached the highest RCA. For each subject sb (sb = a, b, …, l), each potential Q, and each experimental day j, we calculate distance d(uj/ksbQ, uksb/ksbQ). Examples of the distances for subject b are shown in Fig. 9. The horizontal axis of the graph is the distance scale, and the vertical axis of the graph is the ratio scale for the RCAs. The negative correlation between the distances and the RCAs shows high when Q is N1, P2, or P3. For each subject there exists a potential Q such that the correlation is less than −0.5. It is suggested that good-trained brain states could be specified by using such a potential Q. In the same way, we can specify a potential Q such that the absolute value of correlation between the RCAs and the distances is the maximum. The relationship between the RCAs and the distances is shown in Fig. 10, where the symbolic sequences at the rightmost column, just after the right end of the graph, denote subjects, electrode positions for measurement, and peaks (e.g., aPzP1 means subject a, measure position Pz and peak P1).
3.5 Discrimination of the Good-Trained Brain States
We apply the discriminant analysis to the experimental data for 6 peaks and 4 electrodes. We obtain the discriminant function [7] for each subject such that the discrimination ratios are 100 % for all the subjects except for subject i (subject i repeated each task only 6 times). The discrimination ratio of subject i is 83 %.
Each ij-component in Table 3 shows the frequency (the number of times) of being selected by the discriminant analysis. Position Cz is the most frequently selected among all the electrode positions, and peaks N1 and P2 are the most frequently selected ones among all the peaks. However, for all the subjects we cannot remark any particular pair (electrode position and a peak) that are frequently selected together compared with other pairs.
4 Discussions
4.1 Discussions on the Effectiveness of the m-DSAM
The sufficient amount of data for the m-DSAM, is relatively small. For an experiment, an ERP is usually the collection of 1000 data. However, the ERP can be represented by a normal distribution of 6 potentials × 4 parameters (i.e., about 24 data by the m-DSAM). Some of the potentials detected by the m-DSAM cannot be detected by the AM. Although the potentials of the data by the DSAM appear clearly, they are not sharp enough compared with the data by the m-DSAM. Furthermore, the latencies of the data by the DSAM are biased (see N1 and P2 in Fig. 11).
4.2 Discussions on the Discrimination by Eliminating Individual Differences
Adopting the results of the cluster analysis for the RCAs of all the subjects, we can define the good-trained brain states for all the subjects, too. That is, when the RCA of a subject reaches 94.0 % or more, he/she is considered to be at a good-trained brain state. The discrimination defined in this way reflects our educational view point or the traditional learning method.
For each potential Q, let \( {\bar{\mathbf{v}}}_{{ 0 {\text{Q}}}} = \left( {{\mathbf{v}}_{{5_{h} {\text{Q}}}} + {\mathbf{v}}_{{7_{h} {\text{Q}}}} } \right)/2 \). This value is the average of the 5th and the 7th parameters of subject h. Subject h has the highest RCA among all the subject. Using the vector introduced above, we calculate all uj/0Q and the distances d(uj/0Q, u0/0Q) for all the subjects and the electrode positions. Next we find potential Qmax and electrode position E such that the correlation between the distances d(uj/0Qmax, u0/0Qmax) at E and RCAs is the maximum. The relationship between the distances d(uj/0Qmax, u0/0Qmax) at E and the RCA’s for all the subjects is shown in Fig. 12. The meaning of symbolic sequences given in the rightmost part of the graph in Fig. 12 is the same as in Fig. 10.
Using all the distances d(uj/0Q, u0/0Q) in Fig. 12, the discriminant analysis for the good-trained brain states (for all the subjects) is applied to the data. Then we obtain the discrimination ratio 78.7 %.
It is nice that we can discriminate the good-trained brain states for all the subjects from other states by the calculation using the same potentials and the same electrodes. We apply the principal components analysis (PCA) to the data of the distances and the RCAs. In Fig. 13, we show the data plotted in the 2nd component z2 (vertical direction) and the 3rd component z3 (horizontal direction) of the plane. The plotted points in the graph can be classified into four groups (each group is enclosed by an ellipse). In this way we can determine the discrimination ratio for each combination of principal components.
We apply the discriminant analysis to the data for various combinations from the 25 principal components. The following are discrimination ratios for them:
![](http://media.springernature.com/lw472/springer-static/image/chp%3A10.1007%2F978-3-319-20816-9_19/MediaObjects/978-3-319-20816-9_19_Figa_HTML.gif)
For the combinations including z8 (the 8th principle component), which is the principal component highly related to the RCAs, the discrimination ratios are about 97 %. On the other hand, for the combinations of components related only to the distances, the discrimination ratios are at most 87.0 %. These results show that the combinations of various principal components without considering individual differences are useful to determine the good-trained brain states.
5 Conclusions
The problems and the results discussed in this paper are summarized as follows:
-
(1)
We proposed a method called the m-DSAM for translating the original data of ERPs into clear waveforms of the ERPs.
-
(2)
We defined the distances of ERPs between potentials.
-
(3)
By using the cluster analysis, we defined the good-trained brain states for each subject.
-
(4)
By using the distances, we could determine whether a subject is in a good-trained brain state or not.
-
(5)
We defined the good-trained brain states for all the subjects as the states when the RCAs of subjects reach 94.0 % or more.
-
(6)
By the principal component analysis, we extract the combinations of components, which give discrimination ratios more than 80 %.
In Sect. 2.2.3, we introduced a vector vjQ to define the distances between potentials in the waveforms of the ERPs. It would be nice if we could find a suitable vector better than the vector vjQ for our purpose. Such a problem would be worthy for further investigation.
References
Carlson, N.R.: Physiology of Behavior, pp. 27–100. Pearson Education Limited, Edinburgh (2014)
Funada, M., Igarashi, Y., Funada, T., Shibukawa, M., Igarashi, Y., Ninomija, S.P.: Models of event related potentials using the sum of normal distributions and their evaluation by simulations. In: The 16th IASTED International Conference on Applied Simulation and Modelling, Palma De Mallorca, pp. 581–159 (2007)
Funada, M., Shibukawa, M., Igarashi, Y., Funada, T., Ninomija, S.P.: Some characterizations of event-related potentials by reconstructing the distribution of the potentials. Jpn. J. Hum. Factors 46(2), 144–156 (2010)
Funada, M., Igarashi, Y., Funada, T., Shibukawa, M.: A model reflecting the changes of ERPs during iterative learning of calculations. In: The Second IASTED Asian Conference on Modelling, Identification and Control, Phuket, pp. 769–086 (2012)
Funada, M., Funada, T., Shibukawa, M., Akahor, K.: Quantification and analysis of efficiency of iterative learning by using event related potentials. Educ. Technol. Res. 35, 103–113 (2012)
Funada, M., Funada, T., Shibukawa, M.: ERPs composited by multi-data selecting method. In: Proceedings of the 42nd Annual Meeting of the Behaviormetric Society of Japan, pp. 300–301 (2014)
Härdle, W., Simar, L.: Applied Multivariate Statistical Analysis, 2nd edn, pp. 215–247. Springer, Heidelberg (2007)
Luck, S.J.: An Introduction to the Event-Related Potential Technique. The MIT Press, Cambridge (2005)
Luis, F.N.A., Jaime, G.G.: Brain computer interface, a review. Sensors (Basel) 12(2), 1211–1279 (2012)
Minami, S.: Data Processing for Scientific Instrumentation, pp. 84–99. CQ Shuppansha, Tokyo (1992)
Motiya, E.: Operations Research, pp. 55–66. Nihon Rikou Syuppankai, Tokyo (1988)
Nittono, H.: Guidebook Event-Related Potential for Psychology. Ojishobou, Kyoto (2005)
Picton, T.W., Bentin, S., Berg, P., Conchin, E., Hillyard, S.A., Johnson Jr., R., Miller, G.A., Ritter, W., Ruchkin, D.S., Rugg, M.D., Taylor, M.J.: Guideline for using human event-related potentials to study cognition: recording standards and publication criteria. Psychophysiology 37, 128–152 (2000)
Tsuru, N.: Clinical Electroencephalography and Analysis of Electroencephalography. Shinko Igaku Shuppansha, Tokyo (2000)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Funada, M., Funada, T., Igarashi, Y. (2015). Discrimination in Good-Trained Brain States for Brain Computer Interface. In: Schmorrow, D.D., Fidopiastis, C.M. (eds) Foundations of Augmented Cognition. AC 2015. Lecture Notes in Computer Science(), vol 9183. Springer, Cham. https://doi.org/10.1007/978-3-319-20816-9_19
Download citation
DOI: https://doi.org/10.1007/978-3-319-20816-9_19
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-20815-2
Online ISBN: 978-3-319-20816-9
eBook Packages: Computer ScienceComputer Science (R0)