Skip to main content
Springer Nature Link
Log in

Effect of Kernel Size on CNN-Vision-Transformer-Based Gaze Prediction Using Electroencephalography Data

  • Conference paper
  • First Online:
Augmented Cognition (HCII 2024)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14695))

Included in the following conference series:

  • 391 Accesses

Abstract

In this paper, we present an algorithm of gaze prediction from Electroencephalography (EEG) data. EEG-based gaze prediction is a new research topic that can serve as an alternative to traditional video-based eye-tracking. Compared to the existing state-of-the-art (SOTA) method, we improved the root mean-squared-error of EEG-based gaze prediction to 53.06 mm, while reducing the training time to less than 33% of its original duration. Our source code can be found at https://github.com/AmCh-Q/CSCI6907Project.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Bamatraf, S., et al.: A system for true and false memory prediction based on 2d and 3d educational contents and EEG brain signals. Comput. Intell. Neurosci. 2016, 45–45 (2016)

    Article  Google Scholar 

  2. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)

    Google Scholar 

  3. Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

  4. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456. PMLR (2015)

    Google Scholar 

  5. Kastrati, A., et al.: EEGEyeNet: a simultaneous electroencephalography and eye-tracking dataset and benchmark for eye movement prediction. arXiv preprint arXiv:2111.05100 (2021)

  6. Key, M.L., Mehtiyev, T., Qu, X.: Advancing EEG-based gaze prediction using depthwise separable convolution and enhanced pre-processing, preprint (2024)

    Google Scholar 

  7. Khan, S., Naseer, M., Hayat, M., Zamir, S.W., Khan, F.S., Shah, M.: Transformers in vision: a survey. ACM Comput. Surv. (CSUR) 54(10s), 1–41 (2022)

    Article  Google Scholar 

  8. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  9. Lawhern, V.J., Solon, A.J., Waytowich, N.R., Gordon, S.M., Hung, C.P., Lance, B.J.: EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces. J. Neural Eng. 15(5), 056013 (2018)

    Article  Google Scholar 

  10. Ridnik, T., Ben-Baruch, E., Noy, A., Zelnik-Manor, L.: ImageNet-21k pretraining for the masses. arXiv preprint arXiv:2104.10972 (2021)

  11. Schirrmeister, R.T., et al.: Deep learning with convolutional neural networks for EEG decoding and visualization. Hum. Brain Mapp. 38(11), 5391–5420 (2017)

    Article  Google Scholar 

  12. Vaswani, A., et al.: Attention is all you need. Adv. Neural Inf. Process. Syst. 30 (2017)

    Google Scholar 

  13. Wolf, L., et al.: A deep learning approach for the segmentation of electroencephalography data in eye tracking applications. arXiv preprint arXiv:2206.08672 (2022)

  14. Xiang, B., Abdelmonsef, A.: Too fine or too coarse? The goldilocks composition of data complexity for robust left-right eye-tracking classifiers. arXiv preprint arXiv:2209.03761 (2022)

  15. Xiang, B., Abdelmonsef, A.: Vector-based data improves left-right eye-tracking classifier performance after a covariate distributional shift. In: Kurosu, M., et al. (eds.) HCII 2022. LNCS, vol. 13516, pp. 617–632. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-17615-9_44

  16. Yang, R., Modesitt, E.: ViT2EEG: leveraging hybrid pretrained vision transformers for EEG data. arXiv preprint arXiv:2308.00454 (2023)

Download references

Acknowledgments

This study was part of the authors’ work in the course “CSCI 6907 Applied Machine Learning” in The George Washington University.

Author information

Authors and Affiliations

  1. The George Washington University, Washington DC, 20052, USA

    Chuhui Qiu, Bugao Liang & Matthew L. Key

Authors
  1. Chuhui Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bugao Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthew L. Key
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chuhui Qiu .

Editor information

Editors and Affiliations

  1. Soar Technology Inc., Orlando, FL, USA

    Dylan D. Schmorrow

  2. Katmai Government Services, Orlando, FL, USA

    Cali M. Fidopiastis

Ethics declarations

Disclosure of Interests

The authors declare no competing interests.

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Qiu, C., Liang, B., Key, M.L. (2024). Effect of Kernel Size on CNN-Vision-Transformer-Based Gaze Prediction Using Electroencephalography Data. In: Schmorrow, D.D., Fidopiastis, C.M. (eds) Augmented Cognition. HCII 2024. Lecture Notes in Computer Science(), vol 14695. Springer, Cham. https://doi.org/10.1007/978-3-031-61572-6_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-61572-6_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-61571-9

  • Online ISBN: 978-3-031-61572-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics