Skip to main content
SpringerLink
Log in

Music emotion classification and context-based music recommendation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Context-based music recommendation is one of rapidly emerging applications in the advent of ubiquitous era and requires multidisciplinary efforts including low level feature extraction and music classification, human emotion description and prediction, ontology-based representation and recommendation, and the establishment of connections among them. In this paper, we contributed in three distinctive ways to take into account the idea of context awareness in the music recommendation field. Firstly, we propose a novel emotion state transition model (ESTM) to model human emotional states and their transitions by music. ESTM acts like a bridge between user situation information along with his/her emotion and low-level music features. With ESTM, we can recommend the most appropriate music to the user for transiting to the desired emotional state. Secondly, we present context-based music recommendation (COMUS) ontology for modeling user’s musical preferences and context, and for supporting reasoning about the user’s desired emotion and preferences. The COMUS is music-dedicated ontology in OWL constructed by incorporating domain-specific classes for music recommendation into the Music Ontology, which includes situation, mood, and musical features. Thirdly, for mapping low-level features to ESTM, we collected various high-dimensional music feature data and applied nonnegative matrix factorization (NMF) for their dimension reduction. We also used support vector machine (SVM) as emotional state transition classifier. We constructed a prototype music recommendation system based on these features and carried out various experiments to measure its performance. We report some of the experimental results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Allwein E, Schapire R, Singer Y (2000) Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers. Journal of Machine Learning Research 1:113–141

    Article  MathSciNet  Google Scholar 

  2. Boser BE, Guyon I and Vapnik V (1992) A training algorithm for optimal margin classifiers. In proceedings of the Fifth Annual Workshop on Computational Learning Theory : 144–152, ACM Press

  3. Chang C-C and Lin C-J (2001) LIBSVM: a library for support vector machines, Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm of subordinate document. Access 30 Jun 2008.

  4. Cord M, Cunningham P (2008) Machine Learning Techniques for Multimedia. Springer-Verlag Berlin Heidelberg.

  5. Cortes C, Vapnik V (1995) Support-vector network. Machine Learning 20:273–297

    MATH  Google Scholar 

  6. Ellis D, and Poliner G (2007) Identifying `Cover Songs’ with Chroma Features and Dynamic Programming Beat Tracking. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2007) 4:1429–1432

  7. Feng Yazhong , Zhuang, Yueting, Pan Yunhe. (2003) Popular Music Retrieval by Detecting Mood. SIGIR Forum (ACM Special Interest Group on Information Retrieval), Page(s) : 375–376.

  8. Grüninger M, Fox MS (1994) The Role of Mariano Fernández López 4–12 Competency Questions in Enterprise Engineering. IFIP WG 5.7 Workshop on Benchmarking. Theory and Practice. Trondheim, Norway.

  9. Han B, Hwang E, Rho S, Kim M (2007) M-MUSICS: Mobile Content-based Music Retrieval System. ACM Multimedia 2007:496–497

    Google Scholar 

  10. Holzapfel A, Stylianou Y (2008) Musical Genre Classification Using Nonnegative Matrix Factorization-Based Features. IEEE Transactions on Audio, Speech, and Language Processing 16(2):424–434

    Article  Google Scholar 

  11. ISO 226 :2003 Acoustics — Normal equal-loudness level contours.

  12. Jun S, Rho S, Han B, Hwang E (2008) A Fuzzy Inference-based Music Emotion Recognition System. IEEE International Conferences on Visual Information Engineering 2008 (VIE ’08), to appear in Jul. 2008

  13. Juslin PN (2000) Cue utilization in communication of emotion in music performance: Relating performance to perception. J. Experimental Psychology 26:1797–1813

    Google Scholar 

  14. Kalat JW and Shiota MN (2007) Emotion. Thomson. 1/e.

  15. Klapuri A(ed.) and Davy, M(ed.) (2006) Signal Processing Methods for Music Transcription. Springer Science + Business Media LLC.

  16. Krumhansl C (1990) Cognitive Foundations of Musical Pitch. Oxford University Press.

  17. Lee DD, Seung S (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788–791

    Article  Google Scholar 

  18. Lee DD and Seung S (2001) Algorithms for Non-negative Matrix Factorization. Advances in Neural Information Processing System 13 : Proceedings of the 2000 Conference. 556–562. MIT Press.

  19. Lu L, Liu D, Zhang H-J (2006) Automatic MoodDetection and Tracking of Music Audio Signals. IEEE Transactions on Audio, Speech, and Language Processing 14(1):5–18

    Article  MathSciNet  Google Scholar 

  20. Müller M (2008) Information Retrieval for Music and Motion. Springer Berlin Heidelberg New York

  21. Oscar C (2006) Foafing the Music: Bridging the semantic gap in music recommendation. Proceedings of 5th International Semantic Web Conference (ISWC)

  22. Oscar C, Perfecto H, and Xavier S (2006) A multimodal approach to bridge the Music Semantic Gap. International Conference on Semantic and Digital Media Technologies (SAMT)

  23. Pachet F, Casaly D (2000) “A taxonomy of musical genres,” in Proceedings of the 6th Conference on Content-Based Multimedia Information Access (RIAO’00). France, April, Paris

    Google Scholar 

  24. Rabiner LR, Juang B-H (1993) Fundamentals of Speech Recognition. Prentice Hall, New Jersey

    Google Scholar 

  25. Rho S, Han B, Hwang E, Kim M (2008) MUSEMBLE: A Novel Music Retrieval System with Automatic Voice Query Transcription and Reformulation. Journal of Systems and Software 81(7):1065–1080

    Article  Google Scholar 

  26. Robinson DW et al (1956) A re-determination of the equal-loudness relations for pure tones. British Journal of Applied Physics 7:166–181

    Article  Google Scholar 

  27. Russel JA (1980) A circumplex model of affect. Journal of Personality Social Psychology 39:1161–1178

    Article  Google Scholar 

  28. Scherer K (1992) What does facial expression express ? In K.T. Strongman(Ed.) International Review of Studies on Emotions 2 :139–165. Chichester : Wiley.

  29. Scherer KR (2005) What are emotions? And how can they be measures? Social Science Information 44(4):695–729

    Article  Google Scholar 

  30. Schölkopf B, Platt JC et al (1999) Estimating the support of a high-dimensional distribution. Microsoft research corporation technical report MSR-TR-99-87.

  31. Thayer RE (1989) The Biopsychology of Mood and Arousal. Oxford Univ. Press, Oxford, U.K

    Google Scholar 

  32. Xiang H, Ren F, Kuroiwa S, Jiang P (2005) An Experimentation on Creating a Mental State Transition Network. Proceedings of the 2005 IEEE International Confernece on Information Acquisition:432–436

  33. Yazhong Feng; Yueting Zhuang; Yunhe Pan. (2003) Music information retrieval by detecting mood via computational media aesthetics. Web Intelligence, Proceedings. IEEE/WIC International Conference onVolume, Issue, 13–17. Page(s): 235 – 241.

  34. Yves R and Frederick G (2007) Music Ontology Specification, http://www.musicontology.com/

  35. Yves R, Samer A, Mark S, Frederick G (2007) The Music Ontology. Proceedings of the International Conference on Music Information Retrieval, ISMIR 2007:417–422

    Google Scholar 

  36. Zwicker E, Fastl H (1990) Psychoacoustics — Facts and Models. (1st Ed.) Springer.

  37. Kanzaki Music Vocabulary. http://www.kanzaki.com/ns/music

  38. MusicBrainz. http://musicbrainz.org

  39. Music Ontology Specification, http://www.musicontology.com/

  40. COMUS Ontology, http://mil.korea.ac.kr/ontology/0.1/musicontology.owl

  41. All Music Guide, http://www.allmusic.com/

  42. BPM Database, http://www.bpmdatabase.com/

  43. Protégé editor, http://protege.stanford.edu/

  44. Mood Logic, Available at: http://www.moodlogic.com/

Download references

Acknowledgement

This work was supported by the Korea Research Foundation Grant funded by the Korean Government (MOEHRD). (KRF-2007-313-D00758)

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Korea University, Seoul, Korea

    Byeong-jun Han, Sanghoon Jun & Eenjun Hwang

  2. School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, 15213, USA

    Seungmin Rho

Authors
  1. Byeong-jun Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seungmin Rho
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sanghoon Jun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eenjun Hwang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eenjun Hwang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Han, Bj., Rho, S., Jun, S. et al. Music emotion classification and context-based music recommendation. Multimed Tools Appl 47, 433–460 (2010). https://doi.org/10.1007/s11042-009-0332-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-009-0332-6

Keywords

Search

Navigation