Skip to main content
Springer Nature Link
Log in

Incorporating Eye Tracking into an EEG-Based Brainwave Visualization System

  • Conference paper
  • First Online:
Human-Computer Interaction (HCII 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14011))

Included in the following conference series:

  • 1577 Accesses

Abstract

This article describes the incorporation of eye tracking into a brainwaves visualization and analysis system, based on electroencephalography (EEG), to map attention during fruition of audiovisual content. The visualization system was developed in Python, using an Emotiv Insight headset. During the tests, there was a need to identify whether the reactions mapped by the EEG were in fact related to the fruition of the content or whether they originated from elements external to the screen, with the individual looking away and, consequently, losing attention. Based on the Design Science Research methodology, eye tracking was incorporated into the system architecture. For validation, tests were performed with 10 users. Analyzing the generated data, it was possible to identify the correlation between the information presented by the EEG and the gaze of the individuals. In this way, it is possible to increase confidence about the origin of user’s emotions during the fruition of audiovisual content.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Da Silva, T.H.C.T., Cavalcanti, M.D., Becker, V.: Desenvolvimento de um sistema para visualização gráfica de ondas neurais durante consumo de conteúdos midiáticos. IV Jornada Internacional GEMInIS (JIG 2021) (2021)

    Google Scholar 

  2. Becker, V., et al.: A system for graphical visualization of brainwaves to analyse media content consumption. In: Kurosu, M. (eds.) HCII 2022. LNCS, vol. 13303, pp. 319–328. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-05409-9_24

  3. Dresch, A., Lacerda, D.P., Júnior, J.A.V.A.: Design science research: método de pesquisa para avanço da ciência e tecnologia. Bookman Editora (2020)

    Google Scholar 

  4. Järvinen, P.: Action research is similar to design science. Qual. Quant. 41(1), 37–54 (2007)

    Article  Google Scholar 

  5. Hevner, A.R., March, S.T., Park, J.: Design research in information systems research. MIS Q. 28(1), 75–105 (2004)

    Article  Google Scholar 

  6. Toscano, R.M., de Souza, H.B.A.M., da Silva Filho, S.G., Noleto, J.D., Becker, V.: HCI methods and practices for audiovisual systems and their potential contribution to universal design for learning: a systematic literature review. In: Antona, M., Stephanidis, C. (eds.) HCII 2019. LNCS, vol. 11572, pp. 526–541. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23560-4_38

    Chapter  Google Scholar 

  7. Simply User, User Experience Lab. The comparison of accuracy and precision of eye tracking: GazeFlow vs. SMI RED 250 (2013)

    Google Scholar 

  8. Lu, Y., et al.: Combining eye movements and EEG to enhance emotion recognition. In: Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI 2015) (2015)

    Google Scholar 

  9. López-Gil, J.M., et al.: Method for improving EEG based emotion recognition by combining it with synchronized biometric and eye tracking technologies in a non-invasive and low cost way. Front. Comput. Neurosci. (2016)

    Google Scholar 

  10. Xu, S., et al.: Personalized online document, image and video recommendation via commodity eye-tracking. In: RecSys 2008 (2008)

    Google Scholar 

  11. Poole, A., Ball, L.: Eye Tracking in Human-Computer Interaction and Usability Research: Current Status and Future Prospects (2010)

    Google Scholar 

  12. Santos, R., et al.: Eye tracking in neuromarketing: a research agenda for marketing studies. Int. J. Psychol. Stud. 7(1), 32 (2015)

    Google Scholar 

  13. Blascheck, T., et al.: State-of-the-art of visualization for eye tracking data. In: Eurographics Conference on Visualization (EuroVis) (2014)

    Google Scholar 

  14. Farnsworth, B.: 10 Most Used Eye Tracking Metrics and Terms, iMotions (2020)

    Google Scholar 

  15. Emotiv. 2022. Cortex API Getting Started. https://emotiv.gitbook.io/cortex-api/. Accessed 30 Jan 2023

  16. Emotiv. 2022. Insight Manual Technichal Specification. https://emotiv.gitbook.io/insight-manual/introduction/technical-specifications. Accessed 30 Jan 2023

  17. Liao, D., et al.: Design and evaluation of affective virtual reality system based on multimodal physiological signals and self-assessment manikin. IEEE J. Electromagnet. RF Microwaves Med. Biol. 4(3), 216–224 (2020)

    Article  Google Scholar 

  18. Bradley, M.M., Lang, P.J.: Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25(1), 49–59 (1994)

    Article  Google Scholar 

  19. Dattada, V., Mohan, V., Jeevan, M.: Analysis of concealed anger emotion in a neutral speech signal. In: 2019 IEEE International Conference on Distributed Computing, VLSI, Electrical Circuits and Robotics (DISCOVER), pp. 1–5 (2019)

    Google Scholar 

  20. Fransworth, B.: EEG (Electroencephalography): The Complete Pocket Guide (2019). https://imotions.com/blog/eeg/. Accessed 30 Jan 2023

  21. Gabert-Quillen, C.A., Bartolini, E.E., Abravanel, B.T., Sanislow, C.A.: Ratings for emotion film clips. Behav. Res. Methods 47(3), 773–787 (2014). https://doi.org/10.3758/s13428-014-0500-0

    Article  Google Scholar 

  22. Gross, J.J., Levenson, R.W.: Emotion elicitation using films. Cogn. Emot. 9(1), 87–108 (1995)

    Article  Google Scholar 

  23. Samson, A., Kreibig, S., Soderstrom, B., Wade, A., Gross, J.: Eliciting positive, negative, and mixed emotional states: a film library for affective scientists. Cogn. Emot. 30(5), 827–856 (2015)

    Article  Google Scholar 

  24. Schaefer, A., Nils, F., Sanchez, X., Philippot, P.: Assessing the effectiveness of a large database of emotion-eliciting films: a new tool for emotion researchers. Cogn. Emot. 24, 1153–1172 (2010)

    Article  Google Scholar 

  25. Gao, Z., Wang, S.: Emotion recognition from EEG signals using hierarchical Bayesian network with privileged information. In: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval (Shanghai, China) (ICMR 2015), pp. 579–582. Association for Computing Machinery, New York (2015)

    Google Scholar 

  26. Becker, V., Silva, T., Cavalcanti, M., Gambaro, D., Elias, J.: Potencial das interfaces cérebro máquina para a recomendação de conteúdos em sistemas de vídeo sob demanda. In: 4o Congresso Internacional Media Ecology and Image Studies - Reflexões sobre o ecossistema midiático pós pandemia. Ria Editorial, pp. 145–167 (2021)

    Google Scholar 

Download references

Acknowledgments

This work was funded by the Public Call n. 03 Produtividade em Pesquisa PROPESQ/PRPG/UFPB proposal code PVL13414-2020.

Author information

Authors and Affiliations

  1. Laboratory of Interaction and Media, Informatics Center, Federal University of Paraíba, João Pessoa, PB, Brazil

    Matheus Cavalcanti, Felipe Melo, Thiago Silva, Matheus Falcão, Daniel de Queiroz Cavalcanti & Valdecir Becker

Authors
  1. Matheus Cavalcanti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Felipe Melo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thiago Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Matheus Falcão
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Daniel de Queiroz Cavalcanti
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Valdecir Becker
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Valdecir Becker .

Editor information

Editors and Affiliations

  1. The Open University of Japan, Chiba, Japan

    Masaaki Kurosu

  2. Hosei University, Tokyo, Japan

    Ayako Hashizume

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cavalcanti, M., Melo, F., Silva, T., Falcão, M., de Queiroz Cavalcanti, D., Becker, V. (2023). Incorporating Eye Tracking into an EEG-Based Brainwave Visualization System. In: Kurosu, M., Hashizume, A. (eds) Human-Computer Interaction. HCII 2023. Lecture Notes in Computer Science, vol 14011. Springer, Cham. https://doi.org/10.1007/978-3-031-35596-7_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-35596-7_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-35595-0

  • Online ISBN: 978-3-031-35596-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics