Abstract
Identifying the cognitive activities of the human brain is a daunting task due to the fact that those cognitive activities were not observable directly by any known sensing technology. Electroencephalogram (EEG) provided a very useful information that associated with the cognitive activities very closely and was used widely for understanding and recognizing human cognitive activity. In this paper, an experiment was designed to abstractively present the process of “target search” and “action execution” which were among the most common types of cognitive activities during human-computer interaction. EEG signals of the applicants of the experiment were collected and used for subsequent cognitive activity analysis which included a novel temporal-based self-supervised learning approach using BERT to pre-train the data for feature embedding. These encoded points generated by the proposed algorithm could be assigned to the corresponding categories by k-means to capture hidden information about the dominate type of the cognitive activities in any period of 20 ms. The results showed that this approach can distinguish the cognitive activities between the target searching and action executing. And the results suggest that in such a task, subjects’ cognitive activities are relatively pure in the initial search moments, but later in the task, multiple activities may be mixed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Lei, W., Le, D.: Risk evaluation of human factors in flight deck system (2010)
Wang, Y., Liu, X., Zhang, Y., Zhu, Z., Liu, D., Sun, J.: Driving fatigue detection based on EEG signal, pp. 715–718 (2015)
Lin, C.T., Chen, S.A., Ko, L.W., Wang, Y.K.: EEG-based brain dynamics of driving distraction (2011)
Hernández-Rojas, L.G., Martínez, E., Antelis, J.M.: Detection of emergency braking intention using driver’s electroencephalographic signals. IEEE Lat. Am. Trans. 17(01), 111–118 (2019)
Nguyen, T.-H., Chung, W.-Y.: Detection of driver braking intention using EEG signals during simulated driving. Sensors 19(13), 2863 (2019)
Ditthapron, A., Banluesombatkul, N., Ketrat, S., Chuangsuwanich, E., Wilaiprasitporn, T.: Universal joint feature extraction for P300 EEG classification using multi-task autoencoder. IEEE Access 7, 68415–68428 (2019)
Qian, C., Hou, T., Lu, Y., Fu, S.: Affective recognition using EEG signal in human-robot interaction. In: Harris, D. (ed.) EPCE 2018. LNCS (LNAI), vol. 10906, pp. 336–351. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91122-9_29
Zhang, S., Lu, Y., Fu, S.: Recognition of the cognitive state in the visual search task. In: Ayaz, H. (ed.) AHFE 2019. AISC, vol. 953, pp. 363–372. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-20473-0_35
Lee, D.-H., Jeong, J.-H., Kim, K., Yu, B.-W., Lee, S.-W.: Continuous EEG decoding of pilots’ mental states using multiple feature block-based convolutional neural network. IEEE Access 8, 121929–121941 (2020)
Wen, D., et al.: Feature classification method of resting-state EEG signals from amnestic mild cognitive impairment with type 2 diabetes mellitus based on multi-view convolutional neural network. IEEE Trans. Neural Syst. Rehabil. Eng. 28(8), 1702–1709 (2020)
Abbasi, M.U., Rashad, A., Basalamah, A., Tariq, M.: Detection of epilepsy seizures in neo-natal EEG using LSTM architecture. IEEE Access 7, 179074–179085 (2019)
Sheykhivand, S., Mousavi, Z., Rezaii, T.Y., Farzamnia, A.: Recognizing emotions evoked by music using CNN-LSTM networks on EEG signals. IEEE Access 8, 139332–139345 (2020)
Documents of Quick-30 30-Channel Dry EEG Headset. https://d3ccc04c-e3ae-485a-ab07-ad2d43f82fa1.filesusr.com/ugd/ea87ee_3110a3dc3c1a4e77bdf38b9bd7496759.pdf
The Artifact Subspace Reconstruction method of the clean_rawdata plug-in. https://github.com/sccn/clean_rawdata/wiki
Zhao, A., Dong, J., Zhou, H.: Self-supervised learning from multi-sensor data for sleep recognition. IEEE Access 8, 93907–93921 (2020)
Shekhar, R., Jawahar, C.V.: Word Image Retrieval Using Bag of Visual Words (2012)
Cai, Q.: Research on Chinese naming recognition model based on BERT embedding. In: 2019 IEEE 10th International Conference on Software Engineering and Service Science (ICSESS), Beijing, China, pp. 1–4 (2019). https://doi.org/10.1109/ICSESS47205.2019.9040736
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Yang, Y., Zhao, Y., Lu, Y., Fu, S. (2021). Cognitive Activity Recognition Based on Self-supervised Learning from EEG Signals. In: Harris, D., Li, WC. (eds) Engineering Psychology and Cognitive Ergonomics. HCII 2021. Lecture Notes in Computer Science(), vol 12767. Springer, Cham. https://doi.org/10.1007/978-3-030-77932-0_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-77932-0_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-77931-3
Online ISBN: 978-3-030-77932-0
eBook Packages: Computer ScienceComputer Science (R0)