Skip to main content
Springer Nature Link
Log in

An emotion-aware music recommender system: bridging the user’s interaction and music recommendation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In emotion-aware music recommender systems, the user’s current emotion is identified and considered in recommending music to him. We have two motivations to extend the existing systems: (1) to the best of our knowledge, the current systems first estimate the user’s emotions and then suggest music based on it. Therefore, the emotion estimation error affects the recommendation accuracy. (2) Studies show that the pattern of users’ interactions with input devices can reflect their emotions. However, these patterns have not been used yet in emotion-aware music recommender systems. In this study, a music recommender system is proposed to suggest music based on users’ keystrokes and mouse clicks patterns. Unlike the previous ones, the proposed system maps these patterns directly to the user’s favorite music, without labeling its current emotion. The results show that even though this system does not use any additional device, it is highly accurate compared to previous methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. Root Mean Square Error

References

  1. Abdul A, Chen J, Liao H-Y, Chang S-H (2018) An emotion-aware personalized music recommendation system using a convolutional neural networks approach. Appl Sci 8:1103

    Article  Google Scholar 

  2. Aljanaki A, Wiering F, Veltkamp RC (2016) Studying emotion induced by music through a crowdsourcing game. Inform Process Manag 52(1):115–128

    Article  Google Scholar 

  3. Aracena C, Basterrech S, Snáel V, Velásquez J (2015) Neural networks for emotion recognition based on eye tracking data. Paper presented at the 2015 IEEE International Conference on Systems, Man, and Cybernetics

  4. Ayata D, Yaslan Y, Kamasak ME (2018) Emotion based music recommendation system using wearable physiological sensors. IEEE Trans Consumer Electron 64(2):196–203

    Article  Google Scholar 

  5. Burns H, Burns S (2015) Moving Averages 101: Incredible Signals That Will Make You Money in the Stock Market: CreateSpace Independent Publishing Platform

  6. Burns S, Burns H (2017) 5 Moving Average Signals That Beat Buy and Hold: Backtested Stock Market Signals: CreateSpace Independent Publishing Platform

  7. Deng S, Wang D, Li X, Xu G (2015) Exploring user emotion in microblogs for music recommendation. Expert Syst Appl 42(23):9284–9293

    Article  Google Scholar 

  8. Estrada J, Buhia J, Guevarra A, Forcado MR (2017) Keyboard and mouse: tools in identifying emotions during computer activities. Paper presented at the International Conference on Big Data Technologies and Applications

  9. Gavrilescu M (2015) Recognizing emotions from videos by studying facial expressions, body postures and hand gestures. Paper presented at the 2015 23rd Telecommunications Forum Telfor (TELFOR)

  10. Geng B, Li L, Jiao L, Gong M, Cai Q, Wu Y (2015) NNIA-RS: A multi-objective optimization based recommender system. Physica A: Statistic Mech Appl 424:383–397

    Article  MathSciNet  Google Scholar 

  11. Goldberg D, Nichols D, Oki BM, Terry D (1992) Using collaborative filtering to weave an information tapestry. Commun ACM 35:61–70

  12. Gu YH, Yoo SJ, Piao Z, No J, Jiang Z, Yin H (2016) A smart-device news recommendation technology based on the user click behavior. Paper presented at the Sixth International Conference on Emerging Databases: Technologies, Applications, and Theory

  13. Hamedani EM, Kaedi M (2019) Recommending the long tail items through personalized diversification. Knowledge-Based Syst 164:348–357

  14. Hu X, Bai K, Cheng J, Deng J-q, Guo Y, Hu B, . . . Wang F (2017) MeDJ: multidimensional emotion-aware music delivery for adolescent. Paper presented at the Proceedings of the 26th International Conference on World Wide Web Companion

  15. Jannach D, Zanker M, Felfernig A, Friedrich G (2011) Recommender systems, an introduction. Hardback (November 2010)

  16. Kabani H, Khan S, Khan O, Tadvi S (2015) Emotion based music player. Int J Eng Res Gen Sci 3(1):2091–2730

    Google Scholar 

  17. Kawakami A, Furukawa K, Katahira K, Kamiyama K, Okanoya K (2012) Relations between musical structures and perceived and felt emotions. Music Perception: Interdisciplin J 30(4):407–417

    Article  Google Scholar 

  18. Kawakami A, Furukawa K, Katahira K, Okanoya K (2013) Sad music induces pleasant emotion. Frontiers Psychol 4:311

    Article  Google Scholar 

  19. Khan IA, Brinkman WP, Hierons R (2013) Towards estimating computer users’ mood from interaction behaviour with keyboard and mouse. Front Comput Sci 7(6):943–954

    Article  MathSciNet  Google Scholar 

  20. Khanna P, Sasikumar M (2010) Recognising emotions from keyboard stroke pattern. Int J Comput Appl 11(9):1–5

    Google Scholar 

  21. KM AK, Kiran B, Shreyas B, Victor SJ (2015) A multimodal approach to detect user's emotion. Procedia Comput Sci 70:296–303

    Article  Google Scholar 

  22. Lukose S, Upadhya SS (2017) Music player based on emotion recognition of voice signals. Paper presented at the 2017 International Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT)

  23. Nahin ANH, Alam JM, Mahmud H, Hasan K (2014) Identifying emotion by keystroke dynamics and text pattern analysis. Behav Information Technol 33(9):987–996

    Article  Google Scholar 

  24. Pentel A (2017) Emotions and user interactions with Keyboard and Mouse. Paper presented at the 2017 8th International Conference on Information, Intelligence, Systems & Applications (IISA)

  25. Pichl M (2018) Multi-Context-Aware Recommender Systems: A Study on Music Rfecommendation. University of Innsbruck

  26. Pichl M, Zangerle E (2018) Latent feature combination for multi-context music recommendation. Paper presented at the 2018 International Conference on Content-Based Multimedia Indexing (CBMI)

  27. Pichl M, Zangerle E (2020) User models for multi-context-aware music recommendation. Multimed Tools Appl. https://doi.org/10.1007/s11042-020-09890-7

  28. Poirson E, Da Cunha C (2018) A recommender approach based on customer emotions. Expert Syst Appl

  29. Robinson J, Hatten RS (2012) Emotions in music. Music Theory Spectrum 34(2):71–106

    Article  Google Scholar 

  30. Roy S, Biswas M, De D (2020) iMusic: A session-sensitive clustered classical music recommender system using contextual representation learning. Multimed Tools Appl 79:24119–24155

    Article  Google Scholar 

  31. Salmeron-Majadas S, Santos OC, Boticario JG (2014) An evaluation of mouse and keyboard interaction indicators towards non-intrusive and low cost affective modeling in an educational context. Procedia Comput Sci 35:691–700

    Article  Google Scholar 

  32. Sarwar BM, Karypis G, Konstan JA, Riedl J (2001) Item-based collaborative filtering recommendation algorithms. Www 1:285–295

    Article  Google Scholar 

  33. Schedl M, Zamani H, Chen C-W, Deldjoo Y, Elahi M (2018) Current challenges and visions in music recommender systems research. Int J Multimed Information Retriev 7(2):95–116

    Article  Google Scholar 

  34. Shakirova E (2017) Collaborative filtering for music recommender system. Paper presented at the 2017 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus)

  35. Shikder R, Rahaman S, Afroze F, Al Islam AA (2017) Keystroke/mouse usage based emotion detection and user identification. Paper presented at the 2017 International Conference on Networking, Systems and Security (NSysS)

  36. Song K-T, Cervantes C (2017) Music playing system and music playing method based on speech emotion recognition. In: Google Patents

  37. Sulikowski P, Zdziebko T, Turzyński D, Kańtoch E (2018) Human-website interaction monitoring in recommender systems. Procedia Comput Sci 126:1587–1596

    Article  Google Scholar 

  38. Sunitha M, Adilakshmi T (2018) Music recommendation system with user-based and item-based collaborative filtering technique. In Networking Communication and Data Knowledge Engineering (pp 267–278): Springer

  39. Tso K, Schmidt-Thieme L (2006) Attribute-aware collaborative filtering. In From data and information analysis to knowledge engineering (pp 614–621): Springer

  40. Xing B, Zhang K, Sun S, Zhang L, Gao Z, Wang J, Chen S (2015) Emotion-driven Chinese folk music-image retrieval based on DE-SVM. Neurocomput 148:619–627

    Article  Google Scholar 

  41. Yusefi Hafshejani Z, Kaedi M, Fatemi A (2018) Improving sparsity and new user problems in collaborative filtering by clustering the personality factors. Electron Commerce Res 18(4):813–836

  42. Zakamulin V (2017) Market Timing with Moving Averages: The Anatomy and Performance of Trading Rules: Springer

  43. Zentner M, Grandjean D, Scherer KR (2008) Emotions evoked by the sound of music: characterization, classification, and measurement. Emotion 8(4):494

    Article  Google Scholar 

  44. Zhang R, Liu Q-d, Chun-Gui, Wei J-X, Huiyi-Ma (2014) Collaborative filtering for recommender systems. Paper presented at the Second International Conference on Advanced Cloud and Big Data (CBD), Huangshan

    Book  Google Scholar 

  45. Zhou C, Jin Y, Zhang K, Li JYS, Wang X (2018) MusicRoBot: Towards Conversational Context-Aware Music Recommender System. In: Paper presented at the International Conference on Database Systems for Advanced Applications. Gold Coast, Australia

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer Engineering, University of Isfahan, Hezar Jerib Street, Isfahan, 8174673441, Iran

    Saba Yousefian Jazi, Marjan Kaedi & Afsaneh Fatemi

Authors
  1. Saba Yousefian Jazi
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Marjan Kaedi
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Afsaneh Fatemi
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Marjan Kaedi.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yousefian Jazi, S., Kaedi, M. & Fatemi, A. An emotion-aware music recommender system: bridging the user’s interaction and music recommendation. Multimed Tools Appl 80, 13559–13574 (2021). https://doi.org/10.1007/s11042-020-10386-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10386-7

Keywords