Skip to main content
SpringerLink
Log in

A Fine-Grained Domain Adaptation Method for Cross-Session Vigilance Estimation in SSVEP-Based BCI

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14449))

Included in the following conference series:

  • 542 Accesses

Abstract

Brain-computer interface (BCI), a direct communication system between the human brain and external environment, can provide assistance for people with disabilities. Vigilance is an important cognitive state and has a close influence on the performance of users in BCI systems. In this study, a four-target BCI system for cursor control was built based on steady-state visual evoked potential (SSVEP) and twelve subjects were recruited and carried out two long-term BCI experimental sessions, which consisted of two SSVEP-based cursor-control tasks. During each session, electroencephalogram (EEG) signals were recorded. Based on the labeled EEG data of the source domain (previous session) and a small amount of unlabeled EEG data of the target domain (new session), we developed a fine-grained domain adaptation network (FGDAN) for cross-session vigilance estimation in BCI tasks. In the FGDAN model, the graph convolution network (GCN) was built to extract deep features of EEG. The fined-grained feature alignment was proposed to highlight the importance of the different channels figured out by the attention weights mechanism and aligns the feature distributions between source and target domains at the channel level. The experimental results demonstrate that the proposed FGDAN achieved a better performance than the compared methods and indicate the feasibility and effectiveness of our methods for cross-session vigilance estimation of BCI users.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Oken, B.S., Salinsky, M.C., Elsas, S.: Vigilance, alertness, or sustained attention: physiological basis and measurement. Clin. Neurophysiol. 117(9), 1885–1901 (2006)

    Article  Google Scholar 

  2. Zheng, W., et al.: Vigilance estimation using a wearable EOG device in real driving environment. IEEE Trans. Intell. Transp. Syst. 21(1), 170–184 (2020)

    Article  Google Scholar 

  3. Sauvet, F., et al.: In-flight automatic detection of vigilance states using a single EEG channel. IEEE Trans. Biomed. Eng. 61(12), 2840–2847 (2014)

    Article  Google Scholar 

  4. Wolpaw, J., et al.: Brain-computer interfaces for communication and control. Clin. Neurophysiol. 113(6), 767–791 (2002)

    Article  Google Scholar 

  5. Wang, K., et al.: Vigilance estimating in SSVEP-based BCI using multimodal signals. In: 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp. 5974–5978 (2021)

    Google Scholar 

  6. Du, R., Liu, R., Wu, T., Lu, B.: Online vigilance analysis combining video and electrooculography features. In: 2012 International Conference on Neural Information Processing (ICONIP), pp. 447–454 (2012)

    Google Scholar 

  7. Krajewski, J., Batliner, A., Golz, M.: Acoustic sleepiness detection: framework and validation of a speech-adapted pattern recognition approach. Behav. Res. Methods 41(3), 795–804 (2009)

    Article  Google Scholar 

  8. Shi, L., Jiao, Y., Lu, B.: Differential entropy feature for EEG-based vigilance estimation. In: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 6627–6630 (2013)

    Google Scholar 

  9. Shi, L., Lu, B.: EEG-based vigilance estimation using extreme learning machines. Neurocomputing 102, 135–143 (2013)

    Article  Google Scholar 

  10. Zheng, W., Lu, B.: A multimodal approach to estimating vigilance using EEG and forehead EOG. J. Neural Eng. 14, 026017 (2017)

    Article  Google Scholar 

  11. Ko, W., Oh, K., Jeon, E., Suk, H.: VIGNet: a deep convolutional neural network for EEG-based driver vigilance estimation. In: 2020 8th International Winter Conference on Brain-Computer Interface (BCI), pp. 1–3 (2020)

    Google Scholar 

  12. Khessiba, S., Blaiech, A.G., Khalifa, K.B., Abdallah, A.B., Bedoui, M.H.: Innovative deep learning models for EEG-based vigilance detection. Neural Comput. Appl. 33, 6921–6937 (2020)

    Article  Google Scholar 

  13. Zhang, G., Etemad, A.: Capsule attention for multimodal EEG-EOG representation learning with application to driver vigilance estimation. IEEE Trans. Neur. Syst. Rehabil. 29, 1138–1149 (2021)

    Article  Google Scholar 

  14. Jayaram, V., Alamgir, M., Altun, Y., Scholkopf, B., Grosse-Wentrup, M.: Transfer learning in brain-computer interfaces. IEEE Comput. Intell. M. 11(1), 20–31 (2016)

    Article  Google Scholar 

  15. Li, H., Zheng, W., Lu, B.: Multimodal vigilance estimation with adversarial domain adaptation networks. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–6 (2018)

    Google Scholar 

  16. Luo, Y., Lu, B.: Wasserstein-distance-based multi-source adversarial domain adaptation for emotion recognition and vigilance estimation. In: 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 1424–1428 (2021)

    Google Scholar 

  17. Manyakov, N.V., Chumerin, N., Robben, A., Combaz, A., Van Vliet, M., Van Hulle, M.M.: Sampled sinusoidal stimulation profile and multichannel fuzzy logic classification for monitor-based phase-coded SSVEP brain-computer interfacing. J. Neural Eng. 10, 036011 (2013)

    Article  Google Scholar 

  18. Chen, X., Wang, Y., Gao, S., Jung, T.-P., Gao, X.: Filter bank canonical correlation analysis for implementing a high-speed SSVEP-based brain-computer interface. J. Neural Eng. 12, 046008 (2015)

    Article  Google Scholar 

  19. Gomez-Herrero, G., et al.: Automatic removal of ocular artifacts in the EEG without an EOG reference channel. In: Proceedings of the 7th Nordic Signal Processing Symposium, pp. 130–133 (2006)

    Google Scholar 

  20. Jia, Z., et al.: GraphSleepNet: adaptive spatial-temporal graph convolutional networks for sleep stage classification. In: Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI 2020), pp. 1324–1330 (2021)

    Google Scholar 

  21. Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neural networks on graphs with fast localized spectral filtering. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, pp. 3844–3852 (2016)

    Google Scholar 

  22. Gretton, A., et al.: A kernel two-sample test. J. Mach. Learn. Res. 13(25), 723–773 (2012)

    MathSciNet  MATH  Google Scholar 

  23. Hu, J., Shen, L., Albanie, S., Sun, G., Wu, E.: Squeeze-and-excitation networks. IEEE Trans. Pattern Anal. Mach. Intell. 42(8), 2011–2023 (2020)

    Article  Google Scholar 

  24. Shen, F., Dai, G., Lin, G., Zhang, J., Zeng, H.: EEG-based emotion recognition using 4D convolutional recurrent neural network. Cogn. Neurodyn. 14, 815–828 (2020)

    Article  Google Scholar 

  25. Gao, Z., Wang, X., Yang, Y., Li, Y., Ma, K., Chen, G.: A channel-fused dense convolutional network for EEG-based emotion recognition. IEEE T. Cogn. Dev. Syst. 13, 945–954 (2021)

    Article  Google Scholar 

  26. Tzeng, E., et al.: Deep domain confusion: maximizing for domain invariance. arXiv preprint arXiv:1412.3474 (2014)

  27. Long, M., Cao, Y., Wang, J., Jordan, M.: Learning transferable features with deep adaptation networks. In: Proceedings of the 32nd International Conference on Machine Learning, pp. 97–105 (2015)

    Google Scholar 

  28. Ganin, Y., et al.: Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17(59), 1–35 (2016)

    MathSciNet  MATH  Google Scholar 

  29. Chen, X., Wang, S., Wang, J., Long, M.: Representation subspace distance for domain adaptation regression. In: Proceedings of the 38th International Conference on Machine Learning, pp. 1749–1759 (2021)

    Google Scholar 

  30. Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)

    MATH  Google Scholar 

Download references

Acknowledgements

This work was supported by the Beijing Natural Science Foundation [grant numbers 7222311 and J210010], the National Natural Science Foundation of China [grant numbers U21A20388 and 62206285] and China Postdoctoral Science Foundation (grant number 2021M703490).

Author information

Authors and Affiliations

  1. Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China

    Kangning Wang, Minpeng Xu & Dong Ming

  2. Laboratory of Brain Atlas and Brain-Inspired Intelligence, State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Kangning Wang, Shuang Qiu, Wei Wei, Ying Gao & Huiguang He

  3. School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China

    Shuang Qiu & Huiguang He

  4. College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China

    Minpeng Xu & Dong Ming

Authors
  1. Kangning Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuang Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ying Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Huiguang He
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Minpeng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Dong Ming
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuang Qiu .

Editor information

Editors and Affiliations

  1. Central South University, Changsha, China

    Biao Luo

  2. Chinese Academy of Sciences, Beijing, China

    Long Cheng

  3. Zhejiang University, Hangzhou, China

    Zheng-Guang Wu

  4. Guangdong University of Technology, Guangzhou, China

    Hongyi Li

  5. UNSW Sydney, Sydney, NSW, Australia

    Chaojie Li

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, K. et al. (2024). A Fine-Grained Domain Adaptation Method for Cross-Session Vigilance Estimation in SSVEP-Based BCI. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Lecture Notes in Computer Science, vol 14449. Springer, Singapore. https://doi.org/10.1007/978-981-99-8067-3_6

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-8067-3_6

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-8066-6

  • Online ISBN: 978-981-99-8067-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation