Abstract
Epilepsy is one of the common neurological disorders worldwide, which causes significant damage to patients’ health. The EEG clinical manifestations of epilepsy are diverse and complex, and it is necessary to study efficient EEG-based automatic epilepsy detection techniques and use them to monitor and develop epilepsy-related drugs. In this paper, we proposed a convolutional sparse Transformer architecture, where the model can learn directly from the raw EEG data for epilepsy detection and epilepsy-related drug classification. Our proposed model uses a channel attention module to capture the correlation of different spatial locations of the signal. We also construct a sparse Transformer and effectively combine the Transformer and convolutional neural network, which is more suitable for learning on long sequence data like EEG than the standard Transformer, avoiding the performance degradation caused by dense attention. We perform experiments on epilepsy detection and related drug classification datasets, and the results show that the proposed model achieves the current leading performance. The proposed model is a unified architecture suitable for epilepsy detection and drug classification and can also be used for other diseases and drug discovery.
L. Chen—Co-first author.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Engel, J., Jr.: A proposed diagnostic scheme for people with epileptic seizures and with epilepsy: report of the ILAE task force on classification and terminology. Epilepsia 42(6), 796–803 (2001)
Kayser, J., Tenke, C.E.: Issues and considerations for using the scalp surface Laplacian in EEG/ERP research: a tutorial review. Int. J. Psychophysiol. 97(3), 189–209 (2015)
Cilio, M.R.: EEG and the newborn. J Pediatric Neurology 7(1), 25–43 (2009)
Jobert, M., et al.: Guidelines for the recording and evaluation of Pharmaco-EEG data in man: the International Pharmaco-EEG society (IPEG). Neuropsychobiology 66(4), 201–220 (2012)
Skarpaas, T.L., Tcheng, T.K., Morrell, M.J.: Clinical and electrocorticographic response to antiepileptic drugs in patients treated with responsive stimulation. Epilepsy Behav. 83, 192–200 (2018)
Höller, Y., Helmstaedter, C., Lehnertz, K.: Quantitative pharmaco-electroencephalography in antiepileptic drug research. CNS Drugs 32(9), 839–848 (2018)
Hussain, L.: Detecting epileptic seizure with different feature extracting strategies using robust machine learning classification techniques by applying advance parameter optimization approach. Cogn. Neurodyn. 12(3), 271–294 (2018)
Tanveer, M.A., Khan, M.J., Sajid, H., Naseer, N.: Convolutional neural networks ensemble model for neonatal seizure detection. J. Neurosci. Methods 358, 109197 (2021)
Zaremba, W., Sutskever, I., Vinyals, O.: Recurrent neural network regularization arXiv preprint arXiv:1409.2329 (2014)
Affes, A., Mdhaffar, A., Triki, C., Jmaiel, M., Freisleben, B.: A convolutional gated recurrent neural network for epileptic seizure prediction. In: Pagán, J., Mokhtari, M., Aloulou, H., Abdulrazak, B., Cabrera, M.F. (eds.) ICOST 2019. LNCS, vol. 11862, pp. 85–96. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32785-9_8
Han, D., Liu, Q., Fan, W.: A new image classification method using CNN transfer learning and web data augmentation. Expert Syst. Appl. 95, 43–56 (2018)
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022 (2021)
Sun, J., Xie, J., Zhou, H.: EEG classification with transformer-based models. In: 2021 IEEE 3rd Global Conference on Life Sciences and Technologies (LifeTech), pp. 92–93. IEEE (2021)
Winkler, I., Haufe, S., Tangermann, M.: Automatic classifica-tion of artifactual ICA-components for artifact removal in EEG signals. Behav. Brain Funct. 7(1), 1–15 (2011)
Tapani, K.T., Vanhatalo, S., Stevenson, N.J.: Incorporating spike correlations into an SVM-based neonatal seizure detector. Presented at the (2018). https://doi.org/10.1007/978-981-10-5122-7_81
Nagarajan, V., Muralidharan, A., Sriraman, D., et al.: Scalable machine learning architecture for neonatal seizure detection on ultra-edge devices. arXiv preprint arXiv:2111.15569 (2021)
Ansari, A.H., Cherian, P.J., Caicedo, A., Naulaers, G., De Vos, M., Van Huffel, S.: Neonatal seizure detection using deep convolutional neural networks. Int. J. Neural Syst. 29(04), 1850011 (2019)
O’Shea, A., Lightbody, G., Boylan, G., Temko, A.: Neonatal seizure detection from raw multi-channel EEG using a fully convolutional architecture. Neural Netw. 123, 12–25 (2020)
Frassineti, L., Ermini, D., Fabbri, R., Manfredi, C.: Neonatal seizures detection using stationary wavelet transform and deep neural networks: preliminary results. In: 2020 IEEE 20th Mediterranean Electrotechnical Conference (MELECON), pp. 344–349. IEEE (2020)
Louizos, C., Swersky, K., Li, Y., Welling, M., Zemel, R.: The variational fair autoencoder. arXiv preprint arXiv:1511.00830 (2015)
Kalitin, K.Y., Nevzorov, A.A., Spasov, A.A., Sotnikov, P.I.: Deep learning-based i-EEG classification with convolutional neural networks for drug-target interaction prediction. arXiv preprint arXiv:2009.12984 (2020)
Shi, G., Chen, Z., Zhang, R.: A transformer-based spatial-temporal sleep staging model through raw EEG. In: 2021 International Conference on High Performance Big Data and Intelligent Systems (HPBD&IS), pp. 110–115. IEEE (2021)
Wang, Z., Wang, Y., Hu, C., Yin, Z., Song, Y.: Transformers for EEG-based emotion recognition: a hierarchical spatial information learning model. IEEE Sens. J. (2022)
Pedoeem, J., Abittan, S., Yosef, G.B., Keene, S.: Tabs: transformer based seizure detection. In: 2020 IEEE Signal Processing in Medicine and Biology Symposium (SPMB), pp. 1–6. IEEE (2020)
Zaheer, M., et al.: Big bird: transformers for longer sequences. Adv. Neural. Inf. Process. Syst. 33, 17283–17297 (2020)
Roy, A., Saffar, M., Vaswani, A., Grangier, D.: Efficient con-tent-based sparse attention with routing transformers. Trans. Assoc. Comput. Linguist. 9, 53–68 (2021)
Tay, Y., Bahri, D., Yang, L., Metzler, D., Juan, D.-C.: Sparse sinkhorn attention. In International Conference on Machine Learning, pp. 9438–9447. PMLR (2020)
Zhou, H., et al.: Informer: beyond efficient transformer for long sequence time-series forecasting. In: Proceedings of AAAI (2021)
Qin, Z., Zhang, P., Wu, F., Li, X.: FcaNet: frequency channel attention networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 783–792 (2021)
Stevenson, N.J., Tapani, K., Lauronen, L., Vanhatalo, S.: A dataset of neonatal EEG recordings with seizure annotations. Sci. Data 6(1), 1–8 (2019)
Acknowledgements
Supported by grants from the National Natural Science Foundation of China (No.81973182); National Science Foundation of China (No. 61806092); Jiangsu Natural Science Foundation (No. BK20180326); “Double First-Class” University project from China Pharmaceutical University (Program No. CPU2018GF02).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
He, Z. et al. (2023). EEG Convolutional Sparse Transformer for Epilepsy Detection and Related Drug Classification. In: Huang, DS., Premaratne, P., Jin, B., Qu, B., Jo, KH., Hussain, A. (eds) Advanced Intelligent Computing Technology and Applications. ICIC 2023. Lecture Notes in Computer Science, vol 14088. Springer, Singapore. https://doi.org/10.1007/978-981-99-4749-2_63
Download citation
DOI: https://doi.org/10.1007/978-981-99-4749-2_63
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-4748-5
Online ISBN: 978-981-99-4749-2
eBook Packages: Computer ScienceComputer Science (R0)