Skip to main content
SpringerLink
Log in

What Song Am I Thinking Of?

  • Conference paper
  • First Online:
Machine Learning, Optimization, and Data Science (LOD 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14506))

  • 163 Accesses

Abstract

Information Need (IN) is a complex phenomenon due to the difficulty experienced when realising and formulating it into a query format. This leads to a semantic gap between the IN and its representation (e.g., the query). Studies have investigated techniques to bridge this gap by using neurophysiological features. Music Information Retrieval (MIR) is a sub-field of IR that could greatly benefit from bridging the gap between IN and query, as songs present an acute challenge for IR systems. A searcher may be able to recall/imagine a piece of music they wish to search for but still need to remember key pieces of information (title, artist, lyrics) used to formulate a query that an IR system can process. Although, if a MIR system could understand the imagined song, it may allow the searcher to satisfy their IN better. As such, in this study, we aim to investigate the possibility of detecting pieces from Electroencephalogram (EEG) signals captured while participants “listen” to or “imagine” songs. We employ six machine learning models on the publicly available data set, OpenMIIR. In the model training phase, we devised several experiment scenarios to explore the capabilities of the models to determine the potential effectiveness of Perceived and Imagined EEG song data in a MIR system. Our results show that, firstly, we can detect perceived songs using the recorded brain signals, with an accuracy of 62.0% (SD 5.4%). Furthermore, we classified imagined songs with an accuracy of 60.8% (SD 13.2%). Insightful results were also gained from several experiment scenarios presented within this paper. Overall, the encouraging results produced by this study are a crucial step towards information retrieval systems capable of interpreting INs from the brain, which can help alleviate the semantic gap’s negative impact on information retrieval.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Allegretti, M., Moshfeghi, Y., Hadjigeorgieva, M., Pollick, F.E., Jose, J.M., Pasi, G.: When relevance judgement is happening? An EEG-based study. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 719–722 (2015)

    Google Scholar 

  2. Belkin, N.J.: Anomalous states of knowledge as a basis for information retrieval. Can. J. Inf. Sci. 5(1), 133–143 (1980)

    Google Scholar 

  3. Belkin, N.J., Oddy, R.N., Brooks, H.M.: Ask for information retrieval: Part II. Results of a design study. J. Doc. (1982)

    Google Scholar 

  4. Belkin, N.J., Oddy, R.N., Brooks, H.M.: Ask for information retrieval: Part I. Background and theory. J. Doc. 38, 61–71 (1997)

    Google Scholar 

  5. Chen, J.X., Jiang, D.M., Zhang, Y.N.: A hierarchical bidirectional GRU model with attention for EEG-based emotion classification. IEEE Access 7, 118530–118540 (2019). https://doi.org/10.1109/ACCESS.2019.2936817

    Article  Google Scholar 

  6. Chen, X., et al.: Web search via an efficient and effective brain-machine interface. In: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pp. 1569–1572 (2022)

    Google Scholar 

  7. Cole, C.: A theory of information need for information retrieval that connects information to knowledge. J. Am. Soc. Inf. Sci. Technol. 62(7), 1216–1231 (2011). https://doi.org/10.1002/asi.21541, https://onlinelibrary.wiley.com/doi/abs/10.1002/asi.21541

  8. Cole, C.: A theory of information need for information retrieval that connects information to knowledge. J. Am. Soc. Inf. Sci. Technol. 62(7), 1216–1231 (2011)

    Article  Google Scholar 

  9. Davis III, K.M., Kangassalo, L., Spapé, M., Ruotsalo, T.: BrainSourcing: crowdsourcing recognition tasks via collaborative brain-computer interfacing. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–14 (2020)

    Google Scholar 

  10. Eugster, M.J., et al.: Predicting term-relevance from brain signals. In: Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2014, pp. 425–434. Association for Computing Machinery, New York (2014). https://doi.org/10.1145/2600428.2609594

  11. Gao, Z.K., Li, Y.L., Yang, Y.X., Ma, C.: A recurrence network-based convolutional neural network for fatigue driving detection from EEG. Chaos: Interdisc. J. Nonlinear Sci. 29(11), 113126 (2019)

    Article  MathSciNet  Google Scholar 

  12. Gramfort, A., et al.: MEG and EEG data analysis with MNE-python. Front. Neurosci. 267 (2013)

    Google Scholar 

  13. Gwizdka, J., Hosseini, R., Cole, M., Wang, S.: Temporal dynamics of eye-tracking and EEG during reading and relevance decisions. J. Am. Soc. Inf. Sci. 68(10), 2299–2312 (2017)

    Google Scholar 

  14. Hofmann, T.: Collaborative filtering via gaussian probabilistic latent semantic analysis. In: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 259–266 (2003)

    Google Scholar 

  15. Hsu, J.L., Zhen, Y.L., Lin, T.C., Chiu, Y.S.: Personalized music emotion recognition using electroencephalography (EEG). In: 2014 IEEE International Symposium on Multimedia, pp. 277–278 (2014). https://doi.org/10.1109/ISM.2014.19

  16. Huigen, E., Peper, A., Grimbergen, C.: Investigation into the origin of the noise of surface electrodes. Med. Biol. Eng. Compu. 40(3), 332–338 (2002)

    Article  Google Scholar 

  17. Ingwersen, P., Järvelin, K.: The Turn: Integration of Information Seeking and Retrieval in Context. Springer, Heidelberg (2005). https://doi.org/10.1007/1-4020-3851-8

    Book  Google Scholar 

  18. Jacucci, G., et al.: Integrating neurophysiologic relevance feedback in intent modeling for information retrieval. J. Am. Soc. Inf. Sci. 70(9), 917–930 (2019)

    Google Scholar 

  19. Jordan, K.G.: Emergency EEG and continuous EEG monitoring in acute ischemic stroke. J. Clin. Neurophysiol. 21(5), 341–352 (2004)

    Google Scholar 

  20. Kangassalo, L., Spapé, M., Jacucci, G., Ruotsalo, T.: Why do users issue good queries? neural correlates of term specificity. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 375–384 (2019)

    Google Scholar 

  21. Kangassalo, L., Spapé, M., Ravaja, N., Ruotsalo, T.: Information gain modulates brain activity evoked by reading. Sci. Rep. 10(1), 1–10 (2020)

    Article  Google Scholar 

  22. Karameh, F.N., Dahleh, M.A.: Automated classification of EEG signals in brain tumor diagnostics. In: Proceedings of the 2000 American Control Conference. ACC (IEEE Cat. No. 00CH36334), vol. 6, pp. 4169–4173. IEEE (2000)

    Google Scholar 

  23. Kauppi, J.P., et al.: Towards brain-activity-controlled information retrieval: decoding image relevance from meg signals. Neuroimage 112, 288–298 (2015)

    Article  Google Scholar 

  24. Keshavarz, H.: Human information behaviour and design, development and evaluation of information retrieval systems. Program 42(4), 391–401 (2008)

    Article  Google Scholar 

  25. Kingphai, K., Moshfeghi, Y.: On EEG preprocessing role in deep learning effectiveness for mental workload classification. In: Longo, L., Leva, M.C. (eds.) H-WORKLOAD 2021. CCIS, vol. 1493, pp. 81–98. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-91408-0_6

    Chapter  Google Scholar 

  26. Kuhlthau, C.C.: Inside the search process: information seeking from the user’s perspective. J. Am. Soc. Inf. Sci. 42(5), 361–371 (1991)

    Article  Google Scholar 

  27. Maior, H.A., Ramchurn, R., Martindale, S., Cai, M., Wilson, M.L., Benford, S.: fNIRS and neurocinematics. In: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–6 (2019)

    Google Scholar 

  28. Michalkova, D., Parra-Rodriguez, M., Moshfeghi, Y.: Information need awareness: an EEG study. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2022, pp. 610–621. Association for Computing Machinery, New York (2022). https://doi.org/10.1145/3477495.3531999

  29. Midha, S., Maior, H.A., Wilson, M.L., Sharples, S.: Measuring mental workload variations in office work tasks using fNIRS. Int. J. Hum Comput Stud. 147, 102580 (2021)

    Article  Google Scholar 

  30. Morita, Y., Huang, H.H., Kawagoe, K.: Towards music information retrieval driven by EEG signals: architecture and preliminary experiments. In: 2013 IEEE/ACIS 12th International Conference on Computer and Information Science (ICIS), pp. 213–217 (2013). https://doi.org/10.1109/ICIS.2013.6607843

  31. Moshfeghi, Y.: NeuraSearch: neuroscience and information retrieval. In: CEUR Workshop Proceedings, vol. 2950, pp. 193–194 (2021)

    Google Scholar 

  32. Moshfeghi, Y., Pinto, L.R., Pollick, F.E., Jose, J.M.: Understanding relevance: an fMRI study. In: Serdyukov, P., et al. (eds.) ECIR 2013. LNCS, vol. 7814, pp. 14–25. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36973-5_2

    Chapter  Google Scholar 

  33. Moshfeghi, Y., Pollick, F.E.: Search process as transitions between neural states. In: Proceedings of the 2018 World Wide Web Conference, pp. 1683–1692 (2018)

    Google Scholar 

  34. Moshfeghi, Y., Pollick, F.E.: Neuropsychological model of the realization of information need. J. Am. Soc. Inf. Sci. 70(9), 954–967 (2019)

    Google Scholar 

  35. Moshfeghi, Y., Triantafillou, P., Pollick, F.: Towards predicting a realisation of an information need based on brain signals. In: The World Wide Web Conference, pp. 1300–1309 (2019)

    Google Scholar 

  36. Moshfeghi, Y., Triantafillou, P., Pollick, F.E.: Understanding information need: an fMRI study. In: Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2016, pp. 335–344. Association for Computing Machinery, New York (2016). https://doi.org/10.1145/2911451.2911534

  37. Moshfeghi, Y., Triantafillou, P., Pollick, F.E.: Understanding information need: an fMRI study. In: Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 335–344 (2016)

    Google Scholar 

  38. Ntalampiras, S., Potamitis, I.: A statistical inference framework for understanding music-related brain activity. IEEE J. Sel. Top. Signal Process. 13(2), 275–284 (2019). https://doi.org/10.1109/JSTSP.2019.2905431

    Article  Google Scholar 

  39. Paisalnan, S., Moshfeghi, Y., Pollick, F.: Neural correlates of realisation of satisfaction in a successful search process. Proc. Assoc. Inf. Sci. Technol. 58(1), 282–291 (2021)

    Article  Google Scholar 

  40. Paisalnan, S., Pollick, F., Moshfeghi, Y.: Towards understanding neuroscience of realisation of information need in light of relevance and satisfaction judgement. In: Nicosia, G., et al. (eds.) LOD 2021. LNCS, vol. 13163, pp. 41–56. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-95467-3_3

    Chapter  Google Scholar 

  41. Pang, S., Hu, X., Cai, Z., Gong, J., Zhang, M.: Building change detection from bi-temporal dense-matching point clouds and aerial images. Sensors 18(4), 966 (2018)

    Article  Google Scholar 

  42. Ras, Z.W., Wieczorkowska, A.: Advances in Music Information Retrieval, vol. 274. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-11674-2

    Book  Google Scholar 

  43. van Rijsbergen, C.J.: (Invited paper) a new theoretical framework for information retrieval. In: Proceedings of the 9th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 1986, pp. 194–200. Association for Computing Machinery, New York (1986). https://doi.org/10.1145/253168.253208

  44. Savolainen, R.: Information need as trigger and driver of information seeking: a conceptual analysis. Aslib J. Inf. Manage. 69, 2–21 (2017)

    Article  Google Scholar 

  45. Schaefer, R.S., Farquhar, J., Blokland, Y., Sadakata, M., Desain, P.: Name that tune: decoding music from the listening brain. NeuroImage 56(2), 843–849 (2011). https://doi.org/10.1016/j.neuroimage.2010.05.084, https://www.sciencedirect.com/science/article/pii/S1053811910008402,multivariate Decoding and Brain Reading

  46. Schedl, M., Gómez, E., Urbano, J., et al.: Music information retrieval: recent developments and applications. Found. Trends® Inf. Retrieval 8(2–3), 127–261 (2014)

    Google Scholar 

  47. Sonawane, D., Miyapuram, K.P., Rs, B., Lomas, D.J.: GuessTheMusic: song identification from electroencephalography response. In: Proceedings of the 3rd ACM India Joint International Conference on Data Science & Management of Data (8th ACM IKDD CODS & 26th COMAD), pp. 154–162 (2021)

    Google Scholar 

  48. Spape, M., Davis, K., Kangassalo, L., Ravaja, N., Sovijarvi-Spape, Z., Ruotsalo, T.: Brain-computer interface for generating personally attractive images. IEEE Trans. Affect. Comput. 1(1) (2021)

    Google Scholar 

  49. Stober, S.: Toward studying music cognition with information retrieval techniques: Lessons learned from the OpenMIIR initiative. Front. Psychol. 8, 1255 (2017)

    Article  Google Scholar 

  50. Stober, S.: Toward studying music cognition with information retrieval techniques: lessons learned from the OpenMIIR initiative. Front. Psychol. 8 (2017). https://doi.org/10.3389/fpsyg.2017.01255, https://www.frontiersin.org/articles/10.3389/fpsyg.2017.01255

  51. Stober, S., Sternin, A., Owen, A.M., Grahn, J.A.: Towards music imagery information retrieval: introducing the OpenMIIR dataset of EEG recordings from music perception and imagination. In: International Society for Music Information Retrieval Conference (2015)

    Google Scholar 

  52. Taylor, R.S.: The process of asking questions. Am. Doc. 13(4), 391–396 (1962)

    Article  Google Scholar 

  53. Taylor, R.S.: Question-negotiation an information-seeking in libraries. Technical report, Lehigh Univ Bethlehem PA Center for Information Science (1967)

    Google Scholar 

  54. Taylor, R.S.: Question-negotiation and information seeking in libraries. Coll. Res. Libr. 76, 251–267 (1968)

    Article  Google Scholar 

  55. de la Torre-Ortiz, C., Spapé, M.M., Kangassalo, L., Ruotsalo, T.: Brain relevance feedback for interactive image generation. In: Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology, pp. 1060–1070 (2020)

    Google Scholar 

  56. Tzallas, A.T., Tsipouras, M.G., Fotiadis, D.I.: Epileptic seizure detection in EEGs using time-frequency analysis. IEEE Trans. Inf Technol. Biomed. 13(5), 703–710 (2009)

    Article  Google Scholar 

  57. van der Veen, V., dutt-Sharma, N., Cavallaro, L., Bos, H.: Memory errors: the past, the present, and the future. In: Balzarotti, D., Stolfo, S.J., Cova, M. (eds.) RAID 2012. LNCS, vol. 7462, pp. 86–106. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33338-5_5

    Chapter  Google Scholar 

  58. Wissbrock, F.: Information need assessment in information retrieval; beyond lists and queries. In: Proceedings of the 27th German Conference on Artificial Intelligence (2004)

    Google Scholar 

  59. Ye, Z., et al.: Brain topography adaptive network for satisfaction modeling in interactive information access system. In: Proceedings of the 30th ACM International Conference on Multimedia, pp. 90–100 (2022)

    Google Scholar 

  60. Ye, Z., et al.: Towards a better understanding of human reading comprehension with brain signals. In: Proceedings of the ACM Web Conference 2022, pp. 380–391 (2022)

    Google Scholar 

  61. Zhang, J., et al.: Global or local: constructing personalized click models for web search. In: Proceedings of the ACM Web Conference 2022, pp. 213–223 (2022)

    Google Scholar 

  62. Zhao, R., Yan, R., Wang, J., Mao, K.: Learning to monitor machine health with convolutional bi-directional LSTM networks. Sensors 17(2), 273 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. NeuraSearch Laboratory, Department of Computer and Information Sciences, University of Strathclyde, Glasgow, Scotland

    Niall McGuire & Yashar Moshfeghi

Authors
  1. Niall McGuire
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yashar Moshfeghi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Niall McGuire .

Editor information

Editors and Affiliations

  1. University of Catania, Catania, Catania, Italy

    Giuseppe Nicosia

  2. Newcastle University, Newcastle upon Tyne, UK

    Varun Ojha

  3. University of Oxford, Oxford, UK

    Emanuele La Malfa

  4. University of Cambridge, Cambridge, UK

    Gabriele La Malfa

  5. University of Florida, Gainesville, FL, USA

    Panos M. Pardalos

  6. Dana-Farber Cancer Institute, Boston, MA, USA

    Renato Umeton

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

McGuire, N., Moshfeghi, Y. (2024). What Song Am I Thinking Of?. In: Nicosia, G., Ojha, V., La Malfa, E., La Malfa, G., Pardalos, P.M., Umeton, R. (eds) Machine Learning, Optimization, and Data Science. LOD 2023. Lecture Notes in Computer Science, vol 14506. Springer, Cham. https://doi.org/10.1007/978-3-031-53966-4_31

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-53966-4_31

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-53965-7

  • Online ISBN: 978-3-031-53966-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation