Skip to main content
SpringerLink
Log in

Revisiting Neurological Aspects of Relevance: An EEG Study

  • Conference paper
  • First Online:
Machine Learning, Optimization, and Data Science (LOD 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13811))

Abstract

Relevance is a key topic in Information Retrieval (IR). It indicates how well the information retrieved by the search engine meets the user’s information need (IN). Despite research advances in the past decades, the use of brain imaging techniques to investigate complex cognitive processes underpinning relevance is relatively recent, yet has provided valuable insight to better understanding this complex human notion. However, past electrophysiological studies have mainly employed an event-related potential (ERP) component-driven approach. While this approach is effective in exploring known phenomena, it might overlook the key cognitive aspects that significantly contribute to unexplored and complex cognitive processes such as relevance assessment formation. This paper, therefore, aims to study the relevance assessment phenomena using a data-driven approach. To do so, we measured the neural activity of twenty-five participants using electroencephalography (EEG). In particular, the neural activity was recorded in response to participants’ binary relevance assessment (relevant vs. non-relevant) within the context of a Question Answering (Q/A) Task. We found significant variation associated with the user’s subjective assessment of relevant and non-relevant information within the EEG signals associated with P300/CPP, N400 and, LPC components, which confirms the findings of previous studies. Additionally, the data-driven approach revealed neural differences associated with the previously not reported P100 component, which might play important role in early selective attention and working memory modulation. Our findings are an important step towards a better understanding of the cognitive mechanisms involved in relevance assessment and more effective IR systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Recent studies have discussed higher relevance granularity [38, 52], however, it is out of the scope of this paper.

  2. 2.

    To assess the difficulty level, two annotators separately judged question difficulty (i.e. difficult vs easy). The overall inter-annotator agreement was reasonably high (Cohen’s kappa, \(\kappa \) = 0.72).

  3. 3.

    https://sccn.ucsd.edu/wiki/Makoto’s_preprocessing_pipeline.

  4. 4.

    We removed 38 peripheral channels: E1, E8, E14, E17, E21, E25, E32, E38, E43, E44, E48, E49, E56, E57, E63, E64, E68, E69, E73, E74, E81, E82, E88, E89, E94, E95, E99, E100, E107, E113, E114, E119, E120, E121, E125, E126, E127, E128.

  5. 5.

    Region of Interest refers to a selected region of neighbouring electrodes that jointly and significantly contribute towards neurophysiological phenomena of interest.

  6. 6.

    The terms LPC and P600 are commonly interchanged. Relevance assessment has frequently been linked to the P600 ERP component (e.g. [14]. However, the P600 component is mainly associated with ‘syntactic re-analyses’ in language studies. Therefore, the label LPC might be more appropriate to use while focusing on relevance assessment, as the LPC has been linked to memory and recognition processes.

References

  1. Ahmed, L., de Fockert, J.W.: Working memory load can both improve and impair selective attention: evidence from the navon paradigm. Attention Percept. Psychophysics 74(7), 1397–1405 (2012)

    Article  Google Scholar 

  2. Allegretti, M., Moshfeghi, Y., Hadjigeorgieva, M., Pollick, F.E., Jose, J.M., Pasi, G.: When relevance judgement is happening? an EEG-based study. In: SIGIR’15, pp. 719–722. ACM, NY, USA (2015)

    Google Scholar 

  3. Barral, O., et al.: Extracting relevance and affect information from physiological text annotation. User Model. User-Adap. Inter. 26(5), 493–520 (2016). https://doi.org/10.1007/s11257-016-9184-8

    Article  Google Scholar 

  4. Bian, Z., Li, Q., Wang, L., Lu, C., Yin, S., Li, X.: Relative power and coherence of EEG series are related to amnestic mild cognitive impairment in diabetes. Front. Aging Neurosci. 6, 11 (2014)

    Article  Google Scholar 

  5. Bouaffre, S., Faita-Ainseba, F.: Hemispheric differences in the time-course of semantic priming processes: evidence from event-related potentials (ERPS). Brain Cogn. 63(2), 123–135 (2007)

    Article  Google Scholar 

  6. Calbi, M., et al.: How context influences the interpretation of facial expressions: a source localization high-density EEG study on the kuleshov effect. Sci. Rep. 9(1), 1–16 (2019)

    Article  Google Scholar 

  7. Calhoun, V.: Data-driven approaches for identifying links between brain structure and function in health and disease. Dialogues Clin. Neurosci. 20(2), 87 (2018)

    Article  Google Scholar 

  8. Cool, C., Frieder, O., Kantor, P.: Characteristics of text affecting relevance judgments. In: Proceedings of the 14th National Online Meeting 14 (1993)

    Google Scholar 

  9. Curran, T.: Brain potentials of recollection and familiarity. Memory Cognition 28(6), 923–938 (2000)

    Article  Google Scholar 

  10. Delorme, A., Makeig, S.: Eeglab: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 134(1), 9–21 (2004)

    Article  Google Scholar 

  11. Dien, J., Michelson, C.A., Franklin, M.S.: Separating the visual sentence n400 effect from the p400 sequential expectancy effect: cognitive and neuroanatomical implications. Brain Res. 1355, 126–140 (2010)

    Article  Google Scholar 

  12. Dimitriadis, S.I., Salis, C., Tarnanas, I., Linden, D.E.: Topological filtering of dynamic functional brain networks unfolds informative chronnectomics: a novel data-driven thresholding scheme based on orthogonal minimal spanning trees (OMSTS). Front. Neuroinform. 11, 28 (2017)

    Article  Google Scholar 

  13. Elleman, A.M., Oslund, E.L.: Reading comprehension research: implications for practice and policy. Policy Insights Behav. Brain Sci. 6(1), 3–11 (2019)

    Article  Google Scholar 

  14. Eugster, M.J.: Natural brain-information interfaces: recommending information by relevance inferred from human brain signals. Sci. Rep. 6, 38580 (2016)

    Article  Google Scholar 

  15. Eugster, M.J., et al.: Predicting term-relevance from brain signals. In: SIGIR’14, pp. 425–434. ACM, NY, USA (2014)

    Google Scholar 

  16. Farwell, L.A., Donchin, E.: The truth will out: Interrogative polygraphy (lie detection) with event-related brain potentials. Psychophysiology 28(5), 531–547 (1991)

    Article  Google Scholar 

  17. Jacucci, G., et al.: Integrating neurophysiologic relevance feedback in intent modeling for information retrieval. JASIST 70, 917–930 (2019)

    Google Scholar 

  18. Johnson, R., Jr.: The amplitude of the p300 component of the event-related potential: review and synthesis. Adv. Psychophysiol. 3, 69–137 (1988)

    Google Scholar 

  19. Johnson, R., Jr., Donchin, E.: On how p300 amplitude varies with the utility of the eliciting stimuli. Electroencephalogr. Clin. Neurophysiol. 44(4), 424–437 (1978)

    Article  Google Scholar 

  20. Kauppi, J.P.: Towards brain-activity-controlled information retrieval: decoding image relevance from meg signals. Neuroimage 112, 288–298 (2015)

    Article  Google Scholar 

  21. Kelly, S.P., O’Connell, R.G.: Internal and external influences on the rate of sensory evidence accumulation in the human brain. J. Neurosci. 33(50), 19434–19441 (2013)

    Article  Google Scholar 

  22. Kim, H.H., Kim, Y.H.: ERP/MMR algorithm for classifying topic-relevant and topic-irrelevant visual shots of documentary videos. JASIST 70(9), 931–941 (2019)

    Google Scholar 

  23. Laganaro, M., Perret, C.: Comparing electrophysiological correlates of word production in immediate and delayed naming through the analysis of word age of acquisition effects. Brain Topogr. 24(1), 19–29 (2011)

    Article  Google Scholar 

  24. LePendu, P., Dou, D., Frishkoff, G.A., Rong, J.: Ontology database: a new method for semantic modeling and an application to brainwave data. In: Ludäscher, B., Mamoulis, N. (eds.) SSDBM 2008. LNCS, vol. 5069, pp. 313–330. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-69497-7_21

    Chapter  Google Scholar 

  25. Liu, Y., et al.: Early top-down modulation in visual word form processing: Evidence from an intracranial SEEG study. J. Neurosci. 41(28), 6102–6115 (2021)

    Article  Google Scholar 

  26. Luck, S.J.: An Introduction to the Event-related Potential Technique. MIT Press, Cambridge (2014)

    Google Scholar 

  27. Meghdadi, A.H., Karić, M., Berka, C.: EEG analytics: benefits and challenges of data driven eeg biomarkers for neurodegenerative diseases. In: 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), pp. 1280–1285 (2019)

    Google Scholar 

  28. Mognon, A., Jovicich, J., Bruzzone, L., Buiatti, M.: Adjust: an automatic EEG artifact detector based on the joint use of spatial and temporal features. Psychophysiology 48(2), 229–240 (2011)

    Article  Google Scholar 

  29. Moshfeghi, Y.: Neurasearch: neuroscience and information retrieval. In: CEUR Workshop Proceedings, vol. 2950, pp. 193–194 (2021)

    Google Scholar 

  30. Moshfeghi, Y., Pinto, L.R., Pollick, F.E., Jose, J.M.: Understanding relevance: an fMRI study. In: Serdyukov, P., et al. (eds.) ECIR 2013. LNCS, vol. 7814, pp. 14–25. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36973-5_2

    Chapter  Google Scholar 

  31. Moshfeghi, Y., Pollick, F.E.: Search process as transitions between neural states. In: International World Wide Web Conferences Steering Committee, WWW’18, Republic and Canton of Geneva, CHE, pp. 1683–1692 (2018)

    Google Scholar 

  32. Moshfeghi, Y., Pollick, F.E.: Neuropsychological model of the realization of information need. JASIST 70(9), 954–967 (2019)

    Google Scholar 

  33. Moshfeghi, Y., Triantafillou, P., Pollick, F.: Towards predicting a realisation of an information need based on brain signals. In: WWW’19, pp. 1300–1309. ACM, NY, USA (2019)

    Google Scholar 

  34. Moshfeghi, Y., Triantafillou, P., Pollick, F.E.: Understanding information need: an FMRI study. In: SIGIR’16, pp. 335–344. ACM, NY, USA (2016)

    Google Scholar 

  35. O’connell, R.G., Dockree, P.M., Kelly, S.P.: A supramodal accumulation-to-bound signal that determines perceptual decisions in humans. Nat. Neurosci. 15(12), 1729 (2012)

    Article  Google Scholar 

  36. Paisalnan, S., Moshfeghi, Y., Pollick, F.: Neural correlates of realisation of satisfaction in a successful search process. In: Proceedings of the Association for Information Science and Technology, vol. 58, no. 1, pp. 282–291 (2021)

    Google Scholar 

  37. Paisalnan, S., Pollick, F., Moshfeghi, Y.: Towards understanding neuroscience of realisation of information need in light of relevance and satisfaction judgement. In: International Conference on Machine Learning, Optimization, and Data Science, pp. 41–56. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-95467-3_3

  38. Pinkosova, Z., McGeown, W.J., Moshfeghi, Y.: The cortical activity of graded relevance. In: SIGIR’20, pp. 299–308. ACM, NY, USA (2020)

    Google Scholar 

  39. Polich, J.: Updating p300: an integrative theory of p3a and p3b. Clin. Neurophysiol. 118(10), 2128–2148 (2007)

    Article  Google Scholar 

  40. Ruchkin, D., Sutton, S.: Emitted p300 potentials and temporal unvertainty. Electroencephalogr. Clin. Neurophysiol. 45(2), 268–277 (1978)

    Article  Google Scholar 

  41. Rutman, A.M., Clapp, W.C., Chadick, J.Z., Gazzaley, A.: Early top-down control of visual processing predicts working memory performance. J. Cogn. Neurosci. 22(6), 1224–1234 (2010)

    Article  Google Scholar 

  42. Saracevic, T.: Relevance: a review of the literature and a framework for thinking on the notion in information science. part iii: behavior and effects of relevance. JASIST 58(13), 2126–2144 (2007)

    Google Scholar 

  43. Savostyanov, A., Bocharov, A., Astakhova, T., Tamozhnikov, S., Saprygin, A., Knyazev, G.: The behavioral and ERP responses to self-and other-referenced adjectives. Brain Sci. 10(11), 782 (2020)

    Article  Google Scholar 

  44. Schmüser, L., Sebastian, A., Mobascher, A., Lieb, K., Tüscher, O., Feige, B.: Data-driven analysis of simultaneous EEG/FMRI using an ICA approach. Front. Neurosci. 8, 175 (2014)

    Google Scholar 

  45. Sitnikova, T., Salisbury, D.F., Kuperberg, G., Holcomb, P.J.: Electrophysiological insights into language processing in schizophrenia. Psychophysiology 39(6), 851–860 (2002)

    Article  Google Scholar 

  46. Sormunen, E.: Liberal relevance criteria of TREC-: Counting on negligible documents? In: SIGIR’02, pp. 324–330. ACM (2002)

    Google Scholar 

  47. Spironelli, C., Angrilli, A.: Complex time-dependent ERP hemispheric asymmetries during word matching in phonological, semantic and orthographical matching judgment tasks. Symmetry 13(1), 74 (2021)

    Article  Google Scholar 

  48. Tagliabue, C.F., Veniero, D., Benwell, C.S., Cecere, R., Savazzi, S., Thut, G.: The EEG signature of sensory evidence accumulation during decision formation closely tracks subjective perceptual experience. Sci. Rep. 9(1), 1–12 (2019)

    Article  Google Scholar 

  49. Wang, F., et al.: A novel audiovisual brain-computer interface and its application in awareness detection. Sci. Rep. 5(1), 1–12 (2015)

    Google Scholar 

  50. Wang, L., Zheng, J., Huang, S., Sun, H.: P300 and decision making under risk and ambiguity. Comput. Intell. Neurosci. 2015 (2015)

    Google Scholar 

  51. Yang, H., Laforge, G., Stojanoski, B., Nichols, E.S., McRae, K., Köhler, S.: Late positive complex in event-related potentials tracks memory signals when they are decision relevant. Sci. Rep. 9(1), 1–15 (2019)

    Google Scholar 

  52. Zhitomirsky-Geffet, M., Bar-Ilan, J., Levene, M.: How and why do users change their assessment of search results over time? ASIST 52(1), 1–4 (2015)

    Google Scholar 

Download references

Acknowledgement

This work was supported by the Engineering and Physical Sciences Research Council [grant number EP/R513349/1].

Author information

Authors and Affiliations

  1. NeuraSearch Laboratory, University of Strathclyde, Glasgow, UK

    Zuzana Pinkosova & Yashar Moshfeghi

  2. Neuroanalytics Laboratory, University of Strathclyde, Glasgow, UK

    William J. McGeown

Authors
  1. Zuzana Pinkosova
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. William J. McGeown
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yashar Moshfeghi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zuzana Pinkosova .

Editor information

Editors and Affiliations

  1. University of Catania, Catania, Italy

    Giuseppe Nicosia

  2. University of Reading, Reading, UK

    Varun Ojha

  3. University of Oxford, Oxford, UK

    Emanuele La Malfa

  4. University of Cambridge, Cambridge, UK

    Gabriele La Malfa

  5. University of Florida, Gainesville, FL, USA

    Panos Pardalos

  6. Free University of Bozen-Bolzano, Bolzano, Italy

    Giuseppe Di Fatta

  7. University of Catania, Catania, Italy

    Giovanni Giuffrida

  8. Dana-Farber Cancer Institute, Boston, MA, USA

    Renato Umeton

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pinkosova, Z., McGeown, W.J., Moshfeghi, Y. (2023). Revisiting Neurological Aspects of Relevance: An EEG Study. In: Nicosia, G., et al. Machine Learning, Optimization, and Data Science. LOD 2022. Lecture Notes in Computer Science, vol 13811. Springer, Cham. https://doi.org/10.1007/978-3-031-25891-6_41

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-25891-6_41

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-25890-9

  • Online ISBN: 978-3-031-25891-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation