Skip to main content
SpringerLink
Log in

Multimodal mood classification of Hindi and Western songs

  • Published:
Journal of Intelligent Information Systems Aims and scope Submit manuscript

Abstract

Music mood classification is one of the most interesting research areas in music information retrieval, and it has many real-world applications. Many experiments have been performed in mood classification or emotion recognition of Western music; however, research on mood classification of Indian music is still at initial stage due to scarcity of digitalized resources. In the present work, a mood taxonomy is proposed for Hindi and Western songs; both audio and lyrics were annotated using the proposed mood taxonomy. Differences in mood were observed during the annotation of the audio and lyrics for Hindi songs only. The detailed studies on mood classification of Hindi and Western music are presented for the requirement of the recommendation system. LibSVM and Feed-forward neural networks have been used to develop mood classification systems based on audio, lyrics, and a combination of them. The multimodal mood classification systems using Feed-forward neural networks for Hindi and Western songs obtained the maximum F-measures of 0.751 and 0.835, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. https://www.cia.gov/library/publications/the-world-factbook/fields/2098.html.

  2. www.musicir.org/mirex/wiki/MIREXHOME.

  3. http://www.music-ir.org/mirex/wiki/.

  4. www.freemusicarchive.org.

  5. www.mturk.com.

  6. http://www.multimediaeval.org/mediaeval2013/emotion2013/.

  7. http://labrosa.ee.columbia.edu/millionsong/.

  8. http://tdil-dc.in/index.php?option=comvexrtical&parentid=72.

  9. http://www.lyricsmint.com/2011/05/bhaag-dk-bose-aandhi-aayi-delhi-belly.html.

  10. http://www.hindilyrics.net/lyrics/of-Dil%20Duba%20Dil%20Duba.html.

  11. http://www.cs.waikato.ac.nz/ml/weka/.

  12. N-grams refer to one-grams, two-grams and three-grams.

  13. http://ltrc.iiit.ac.in/analyzer/hindi/.

  14. http://www.rednoise.org/rita/.

  15. https://developer.spotify.com/web-api/get-audio-features/.

References

  • Abburi, H., Akkireddy, E.S.A., Gangashetty, S.V., & Mamidi, R. (2016). Multimodal sentiment analysis of telugu songs. In Proceedings of the 4th workshop on sentiment analysis where AI meets psychology (SAAIP 2016) (pp. 48–52).

  • Bertin-Mahieux, T., Ellis, D.P.W., Whitman, B., & Lamere, P. (2011). The million song dataset. In Proceedings of the 12th international society for music information retrieval conference (ISMIR 2011) (pp. 591–596).

  • Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46.

    Article  Google Scholar 

  • Duncan, N., & Fox, M. (2005). Computer–aided music distribution: the future of selection, retrieval and transmission. First Monday, 10(4). https://doi.org/10.5210/fm.v10i4.1220.

  • Eyben, F., Wöllmer, M., & Schuller, B. (2010). Opensmile: the munich versatile and fast open-source audio feature extractor. In Proceedings of the 18th ACM international conference on Multimedia (pp. 1459–1462).

  • Hall, M.A. (1999). Correlation-based feature selection for machine learning. PhD dissertation: The University of Waikato.

    Google Scholar 

  • Hampiholi, V. (2012). A method for music classification based on perceived mood detection for Indian bollywood music. International Journal of Computer, Electrical, Automation, Control and Information Engineering, 6(12), 1636–1643.

    Google Scholar 

  • Hevner, K. (1936). Experimental studies of the elements of expression in music. The American Journal of Psychology, 48(2), 246–268.

    Article  Google Scholar 

  • Hu, X. (2010). Music and mood: where theory and reality meet. In Proceedings of the iConference 2010.

  • Hu, X., & Downie, J.S. (2010a). Improving mood classification in music digital libraries by combining lyrics and audio. In Proceedings of the 10th annual joint conference on digital libraries (pp. 159–168).

  • Hu, X., & Downie, J.S. (2010b). When lyrics outperform audio for music mood classification: a feature analysis. In Proceedings of the 11th international society for music information retrieval conference (ISMIR 2010) (pp. 619–624).

  • Hu, X., Downie, J.S., Laurier, C., Bay, M., & Ehmann, A.F. (2008). The 2007 MIREX audio mood classification task: lessons learned. In Proceedings of the 9th international society for music information retrieval conference (ISMIR 2008) (pp. 462–467).

  • Hu, X., Choi, K., & Downie, J.S. (2017). A framework for evaluating multimodal music mood classification. Journal of the Association for Information Science and Technology, 68(2), 273–285. https://doi.org/10.1002/asi.23649.

    Article  Google Scholar 

  • Katayose, H., Imai, M., & Inokuchi, S. (1988). Sentiment extraction in music. In Proceedings of the 9th international conference on pattern recognition (pp. 1083–1087).

  • Kim, Y.E., Schmidt, E.M., & Emelle, L. (2008). MoodSwings: a collaborative game for music mood label collection. In Proceedings of the 9th international society for music information retrieval conference (ISMIR 2008) (pp. 231–236).

  • Kim, Y.E., Schmidt, E.M., Migneco, R., Morton, B.G., Richardson, P., Scott, J., Speck, J.A., & Turnbull, D. (2010). Music emotion recognition: a state of the art review. In Proceedings of the 11th international society for music information retrieval conference (ISMIR 2010) (pp. 255–266).

  • Koduri, G.K., & Indurkhya, B. (2010). A behavioral study of emotions in south Indian classical music and its implications in music recommendation systems. In Proceedings of the 2010 ACM workshop on social, adaptive and personalized multimedia interaction and access (pp. 55–60).

  • Lamere, P. (2008). Social tagging and music information retrieval. Journal of New Music Research, 37(2), 101–114.

    Article  Google Scholar 

  • Lang, P.J., Bradley, M.M., & Cuthbert, B.N. (1998). Emotion, motivation, and anxiety: Brain mechanisms and psychophysiology. Biological Psychiatry, 44(12), 1248–1263.

    Article  Google Scholar 

  • Laurier, C., & Herrera, P. (2007). Audio music mood classification using support vector machine. MIREX task on Audio Mood Classification, 2–4.

  • Laurier, C., Grivolla, J., & Herrera, P. (2008). Multimodal music mood classification using audio and lyrics. In Proceedings of the 7th international conference on machine learning and applications (ICMLA’08) (pp. 688–693).

  • Laurier, C., Sordo, M., Serra, J., & Herrera, P. (2009). Music mood representations from social tags. In Proceedings of the 10th international society for music information retrieval conference (ISMIR 2009) (pp. 381–386).

  • Liu, D., Lu, L., & Zhang, H. (2003). Automatic mood detection from acoustic music data. In Proceedings of the 6th international conference on music information retrieval (ISMIR-2003) (pp. 81–87).

  • Lu, L., Liu, D., & Zhang, H.J. (2006). Automatic mood detection and tracking of music audio signals. IEEE Transactions on Audio, Speech, and Language Processing, 14(1), 5–18.

    Article  Google Scholar 

  • Mathematica Neural NetworksTrain and Analyze Neural Networks to Fit Your Data. 2005. Wolfram Research Inc., First Edition, Champaign, Illinois, USA.

  • Mayer, R., Neumayer, R., & Rauber, A. (2008). Combination of audio and lyrics features for genre classification in digital audio collections. In Proceedings of the 16th ACM international conference on multimedia (pp. 159–168).

  • McKay, C., Fujinaga, I., & Depalle, P. (2005). jAudio: a feature extraction library. In Proceedings of the 6th international conference on music information retrieval (pp. 600–603).

  • Patra, B.G., Das, D., & Bandyopadhyay, S. (2013a). Automatic music mood classification of Hindi songs. In Proceedings of the 3rd workshop on sentiment analysis where AI meets psychology (SAAIP 2013) (pp. 24–28).

  • Patra, B.G., Das, D., & Bandyopadhyay, S. (2013b). Unsupervised approach to Hindi music mood classification. In Proceedings of the mining intelligence and knowledge exploration (pp. 62–69).

    Chapter  Google Scholar 

  • Patra, B.G., Maitra, P., Das, D., & Bandyopadhyay, S. (2015a). MediaEval 2015: music emotion recognition based on feed-forward neural network. In Proceedings of MediaEval 2015 Workshop.

  • Patra, B.G., Das, D., & Bandyopadhyay, S. (2015b). Music emotion recognition system. In Proceedings of the international symposium frontiers of research speech and music (FRSM-2015) (pp. 114–119).

  • Patra, B.G., Das, D., & Bandyopadhyay, S. (2015c). Mood classification of Hindi songs based on lyrics. In Proceedings of the 12th international conference on natural language processing (ICON-2015) (pp. 261–267).

  • Patra, B.G., Das, D., & Bandyopadhyay, S. (2016a). Multimodal mood classification framework for Hindi songs. Computación y Sistemas, 20(3), 515–526.

    Article  Google Scholar 

  • Patra, B.G., Das, D., & Bandyopadhyay, S. (2016b). Labeling data and developing supervised framework for Hindi music mood analysis. Journal of Intelligent Information Systems, 48(3), 633–651. https://doi.org/10.1007/s10844-016-0436-1.

    Article  Google Scholar 

  • Patra, B.G., Das, D., & Bandyopadhyay, S. (2016c). Multimodal mood classification - a case study of differences in Hindi and Western songs. In Proceedings of the 26th international conference on computational linguists (COLING-2016) (pp. 1980–1989).

  • Posner, J., Russell, J.A., & Peterson, B.S. (2005). The circumplex model of affect: an integrative approach to affective neuroscience, cognitive development, and psychopathology. Development and Psychopathology, 17(03), 715–734.

    Article  Google Scholar 

  • Rumelhart, D.E., Hinton, G.E., & Williams, R.J. (1986). Learning representations by back-propagating errors. Nature, 323, 533–536.

    Article  Google Scholar 

  • Russell, J.A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161–1178.

    Article  Google Scholar 

  • Soleymani, M., Caro, M.N., Schmidt, E.M., Sha, C.Y., & Yang, Y.H. (2013). 1000 songs for emotional analysis of music. In Proceedings of the 2nd ACM international workshop on crowdsourcing for multimedia (pp. 1–6).

  • Thayer, R.E. (1990). The biopsychology of mood and arousal. Oxford University Press.

  • Ujlambkar, A.M. (2012). Automatic mood classification of Indian popular music Master’s Thesis. College of Engineering, Pune.

  • Velankar, M.R., & Sahasrabuddhe, H.V. (2012). A pilot study of Hindustani music sentiments. In Proceedings of the 2nd workshop on sentiment analysis where AI meets psychology (SAAIP-2012) (pp. 91–98).

  • Watson, D., Wiese, D., Vaidya, J., & Tellegen, A. (1999). The two general activation systems of affect: structural findings, evolutionary considerations, and psychobiological evidence. Journal of Personality and Social Psychology, 76(5), 820–838.

    Article  Google Scholar 

  • Yang, Y.H., Lin, Y.C., Cheng, H.T., Liao, I-Bin, & Ho, Y.C. (2008). Toward multi-modal music emotion classification. In Proceedings of the pacific-rim conference on multimedia (pp. 70–79).

    Chapter  Google Scholar 

  • Zaanen, M.V., & Kanters, P. (2010). Automatic mood classification using TF*IDF based on lyrics. In Proceedings of the 11th international society for music information retrieval conference (ISMIR 2010) (pp. 75–80).

Download references

Acknowledgements

The work reported in this paper is supported by a grant from the “Visvesvaraya Ph.D. Scheme for Electronics and IT” funded by Media Lab Asia of Ministry of Electronics and Information Technology (MeitY), Government of India. The authors are thankful to Afif Ahmed, Anit, Arijit Das, and Niloy Mukherjee, for helping in data collection. The authors are also thankful to the anonymous reviewers for their helpful comments.

Author information

Authors and Affiliations

  1. School of Biomedical Informatics, UTHealth, Houston, TX, USA

    Braja Gopal Patra

  2. Department of Computer Science and Engineering, Jadavpur University, Kolkata, India

    Dipankar Das & Sivaji Bandyopadhyay

Authors
  1. Braja Gopal Patra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dipankar Das
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sivaji Bandyopadhyay
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Braja Gopal Patra.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Patra, B.G., Das, D. & Bandyopadhyay, S. Multimodal mood classification of Hindi and Western songs. J Intell Inf Syst 51, 579–596 (2018). https://doi.org/10.1007/s10844-018-0497-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10844-018-0497-4

Keywords

Search

Navigation