Skip to main content
SpringerLink
Log in

MMusic: a hierarchical multi-information fusion method for deep music recommendation

  • Research
  • Published:
Journal of Intelligent Information Systems Aims and scope Submit manuscript

Abstract

With the explosive growth of music volume, music recommendation systems have become an important tool for online music platforms to alleviate the information overload problem. Through the use of deep learning, the multi-information fusion-based deep recommendation method has gained popularity in the field of music recommendation systems research. However, most existing studies only consider the different kinds of information of users or music and fail to capture information’s internal and external associations. In this work, we propose a hierarchical multi-information fusion method for deep music recommendation (MMusic), to fully exploit the features of each type of information and to better learn the representation of users and music. Specifically, combined with the features of music recommendation, we identify various kinds of information describing users and music, respectively. Then, we learn about the interactions within and between different kinds of information for fusion. We conduct extensive experiments on the publicly available dataset NOWPLAYINGRS. The results show that MMusic achieves the best performance compared with the baselines, which verifies the effectiveness and rationality of our model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability

Data will be made available on request.

Notes

  1. https://developer.spotify.com/documentation/web-api/reference/#/operations/get-audio-features

  2. Dataset available from https://zenodo.org/record/3247476#.Yhnb7ehBybh

  3. https://github.com/Dolly0209/MMusic

References

  • Alharbi, N., & Caragea, D. (2021). Cross-domain attentive sequential recommendations based on general and current user preferences (cd-asr). In IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (pp. 48–55). Association for Computing Machinery. https://doi.org/10.1145/3486622.3493949.

  • Ben Sassi, I., & Ben Yahia, S. (2021). How does context influence music preferences: a user-based study of the effects of contextual information on users’ preferred music. Multimedia Systems, 27, 143–160. https://doi.org/10.1007/s00530-020-00717-x

    Article  Google Scholar 

  • Cami, B. R., Hassanpour, H., & Mashayekhi, H. (2019). User preferences modeling using dirichlet process mixture model for a content-based recommender system. Knowledge-Based Systems, 163, 644–655. https://doi.org/10.1016/j.knosys.2018.09.028

    Article  Google Scholar 

  • Cao, Y., & Liu, P. (2022). Personalized music hybrid recommendation algorithms fusing gene features. Mathematical Problems in Engineering, 2022. https://doi.org/10.1155/2022/9209022.

  • de Assunção, W. G., & Zaina, L. A. M. (2022). Evaluating user experience in music discovery on deezer and spotify. In Proceedings of the 21st Brazilian Symposium on Human Factors in Computing Systems (pp. 1–11). Association for Computing Machinery. https://doi.org/10.1145/3554364.3560901.

  • Gómez-Cañón, J. S., Gutiérrez-Páez, N., Porcaro, L., Porter, A., Cano, E., Herrera-Boyer, P., Gkiokas, A., Santos, P., Hernández-Leo, D., Karreman, C. et al. (2022). Trompa-mer: an open dataset for personalized music emotion recognition. Journal of Intelligent Information Systems, (pp. 1–22). https://doi.org/10.1007/s10844-022-00746-0.

  • Guo, L., Yin, H., Wang, Q., Chen, T., Zhou, A., & Quoc Viet Hung, N. (2019). Streaming session-based recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (pp. 1569–1577). Association for Computing Machinery. https://doi.org/10.1145/3292500.3330839.

  • Hidasi, B., Karatzoglou, A., Baltrunas, L., & Tikk, D. (2015). Session-based recommendations with recurrent neural networks. arXiv:1511.06939

  • Katharopoulos, A., Vyas, A., Pappas, N., & Fleuret, F. (2020). Transformers are rnns: Fast autoregressive transformers with linear attention. In International Conference on Machine Learning (pp. 5156–5165). PMLR. arXiv:2006.16236

  • Kim, H.-G., Kim, G. Y., & Kim, J. Y. (2019). Music recommendation system using human activity recognition from accelerometer data. IEEE Transactions on Consumer Electronics, 65, 349–358. https://doi.org/10.1109/TCE.2019.2924177

    Article  Google Scholar 

  • Kulkarni, S., & Rodd, S. F. (2020). Context aware recommendation systems: A review of the state of the art techniques. Computer Science Review, 37, 100255. https://doi.org/10.1016/j.cosrev.2020.100255.

  • Li, J., Ren, P., Chen, Z., Ren, Z., Lian, T., & Ma, J. (2017). Neural attentive session-based recommendation. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (pp. 1419–1428). Association for Computing Machinery. https://doi.org/10.1145/3132847.3132926.

  • Marukatat, S. (2022). Tutorial on pca and approximate pca and approximate kernel pca. Artificial Intelligence Review, (pp. 1–33). https://doi.org/10.1007/s10462-022-10297-z.

  • Nam, J., Choi, K., Lee, J., Chou, S.-Y., & Yang, Y.-H. (2018). Deep learning for audio-based music classification and tagging: Teaching computers to distinguish rock from bach. IEEE Signal Processing Magazine, 36, 41–51. https://doi.org/10.1109/MSP.2018.2874383

    Article  Google Scholar 

  • Pichl, M., & Zangerle, E. (2021). User models for multi-context-aware music recommendation. Multimedia Tools and Applications, 80, 22509–22531. https://doi.org/10.1007/s11042-020-09890-7

    Article  Google Scholar 

  • Porcaro, L., Gómez, E., & Castillo, C. (2022). Diversity in the music listening experience: Insights from focus group interviews. In ACM SIGIR Conference on Human Information Interaction and Retrieval (pp. 272–276). Association for Computing Machinery. https://doi.org/10.1145/3498366.3505778.

  • Pulis, M., & Bajada, J. (2021). Siamese neural networks for content-based cold-start music recommendation. In Proceedings of the 15th ACM Conference on Recommender Systems. Association for Computing Machinery. https://doi.org/10.1145/3460231.3478847.

  • Ras, Z. W., & Wieczorkowska, A. (2010). Advances in Music Information Retrieval (vol. 274). Springer. http://www.springer.com/engineering/book/978-3-642-11673-5.

  • Ras, Z. W., Wieczorkowska, A., & Tsumoto, S. (2021). Recommender Systems for Medicine and Music. Springer.

  • Rashed, A., Elsayed, S., & Schmidt-Thieme, L. (2022). Carca: Context and attribute-aware next-item recommendation via cross-attention. arXiv:2204.06519.

  • Rendle, S., Freudenthaler, C., Gantner, Z., & Schmidt-Thieme, L. (2012). Bpr: Bayesian personalized ranking from implicit feedback. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence. AUAI Press. arXiv:1205.2618.

  • Sachdeva, N., Gupta, K., & Pudi, V. (2018). Attentive neural architecture incorporating song features for music recommendation. In Proceedings of the 12th ACM Conference on Recommender Systems (pp. 417–421). Association for Computing Machinery. https://doi.org/10.1145/3240323.3240397.

  • Sánchez-Moreno, D., González, A. B. G., Vicente, M. D. M., Batista, V. F. L., & García, M. N. M. (2016). A collaborative filtering method for music recommendation using playing coefficients for artists and users. Expert Systems with Applications, 66, 234–244. https://doi.org/10.1016/j.eswa.2016.09.019

    Article  Google Scholar 

  • Sassi, I. B., Yahia, S. B., & Liiv, I. (2021). Morec: At the crossroads of context-aware and multi-criteria decision making for online music recommendation. Expert Systems with Applications, 183,. https://doi.org/10.1016/j.eswa.2021.115375

  • Sheikh Fathollahi, M., & Razzazi, F. (2021). Music similarity measurement and recommendation system using convolutional neural networks. International Journal of Multimedia Information Retrieval, 10, 43–53. https://doi.org/10.1007/s13735-021-00206-5

    Article  Google Scholar 

  • Tommasel, A., Rodriguez, J. M., & Godoy, D. (2022). Haven’t i just listened to this?: Exploring diversity in music recommendations. In Adjunct Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization (pp. 35–40). Association for Computing Machinery. https://doi.org/10.1145/3511047.3536409.

  • Vall, A., Dorfer, M., Eghbal-Zadeh, H., Schedl, M., Burjorjee, K., & Widmer, G. (2019). Feature-combination hybrid recommender systems for automated music playlist continuation. User Modeling and User-Adapted Interaction, 29, 527–572. https://doi.org/10.1007/s11257-018-9215-8

    Article  Google Scholar 

  • Vančura, V. (2021). Neural basket embedding for sequential recommendation. In Proceedings of the 15th ACM Conference on Recommender Systems (pp. 878–883). Association for Computing Machinery. https://doi.org/10.1145/3460231.3473896.

  • Vystrčilová, M., & Peška, L. (2020). Lyrics or audio for music recommendation? In Proceedings of the 10th International Conference on Web Intelligence, Mining and Semantics (pp. 190–194). Association for Computing Machinery. https://doi.org/10.1145/3405962.3405963

  • Wang, X., He, X., Wang, M., Feng, F., & Chua, T.-S. (2019). Neural graph collaborative filtering. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 165–174). Association for Computing Machinery. https://doi.org/10.1145/3331184.3331267.

  • Wang, Z., Wei, W., Cong, G., Li, X.-L., Mao, X.-L., & Qiu, M. (2020). Global context enhanced graph neural networks for session-based recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 169–178). Association for Computing Machinery. https://doi.org/10.1145/3397271.3401142.

  • Yousefian Jazi, S., Kaedi, M., & Fatemi, A. (2021). An emotion-aware music recommender system: bridging the users’ interaction and music recommendation. Multimedia Tools and Applications, 80, 13559–13574. https://doi.org/10.1007/s11042-020-10386-7

    Article  Google Scholar 

  • Yuan, F., Karatzoglou, A., Arapakis, I., Jose, J. M., & He, X. (2019). A simple convolutional generative network for next item recommendation. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining (pp. 582–590). Association for Computing Machinery. https://doi.org/10.1145/3289600.3290975

  • Zangerle, E., Pichl, M., Gassler, W., & Specht, G. (2014). # nowplaying music dataset: Extracting listening behavior from twitter. In Proceedings of the First International Workshop on Internet-scale Multimedia Management (pp. 21–26). Association for Computing Machinery. https://doi.org/10.1145/2661714.2661719.

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Nos. 72271024,71871019).

Author information

Authors and Affiliations

  1. Department of Management Science and Engineering, School of Economics and Management, University of Science and Technology Beijing, Beijing, 100083, China

    Jing Xu, Mingxin Gan & Xiongtao Zhang

Authors
  1. Jing Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingxin Gan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiongtao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Jing Xu: Conceptualization, Methodology, Data Curation, Software, Validation, Writing - Original Draft, Writing - review. Mingxin Gan: Conceptualization, Writing - review and editing, Supervision, Funding acquisition. Xiongtao Zhang: Conceptualization, Writing - review and editing, Supervision.

Corresponding author

Correspondence to Mingxin Gan.

Ethics declarations

Ethics Approval

Not Applicable.

Conflicts of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, J., Gan, M. & Zhang, X. MMusic: a hierarchical multi-information fusion method for deep music recommendation. J Intell Inf Syst 61, 795–818 (2023). https://doi.org/10.1007/s10844-023-00786-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10844-023-00786-0

Keywords

Search

Navigation