skip to main content
10.1145/3555776.3577853acmconferencesArticle/Chapter ViewAbstractPublication PagessacConference Proceedingsconference-collections
poster

Graph Convolutional Neural Network for Multimodal Movie Recommendation

Published: 07 June 2023 Publication History

Abstract

The Recommendation System (RS) development and recommending customers' preferred products to the customer are highly desirable motives in today's digital market. Most of the RSs are mainly based on textual information of the engaged entities in the platform and the ratings provided by the users to the products. This paper develops a movie recommendation system where the cold-start problem relating to rating information dependency has been dealt with and the multi-modality approach is introduced. The proposed method differs from existing approaches in three main aspects: (a) implementation of knowledge graph for text embedding, (b) besides textual information, other modalities of movies like video, and audio are employed rather than rating information for generating movie/user representation and this approach deals with the cold-start problem effectively, (c) utilization of graph convolutional network (GCN) for generating some further hidden features and also for developing regression system.

References

[1]
Gediminas Adomavicius and YoungOk Kwon. 2007. New Recommendation Techniques for Multicriteria Rating Systems. IEEE Intelligent Systems 22, 3 (2007), 48--55.
[2]
Mohammed Fadhel Aljunid and Manjaiah Doddaghatta Huchaiah. 2020. Multi-model deep learning approach for collaborative filtering recommendation system. CAAI Transactions on Intelligence Technology 5, 4 (2020), 268--275.
[3]
Mohammed Fadhel Aljunid and DH Manjaiah. 2019. Movie recommender system based on collaborative filtering using apache spark. In Data management, analytics and innovation. Springer, 283--295.
[4]
Julio Barbieri, Leandro GM Alvim, Filipe Braida, and Geraldo Zimbrão. 2017. Autoencoders and recommender systems: COFILS approach. Expert Systems with Applications 89 (2017), 81--90.
[5]
Tianfeng Chai and Roland R Draxler. 2014. Root mean square error (RMSE) or mean absolute error (MAE). Geoscientific Model Development Discussions 7, 1 (2014), 1525--1534.
[6]
Daipayan Chakder, Prabir Mondal, Subham Raj, Sriparna Saha, Angshuman Ghosh, and Naoyuki Onoe. 2022. Graph Network based Approaches for Multimodal Movie Recommendation System. In 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 409--414.
[7]
Aminu Da'u, Naomie Salim, Idris Rabiu, and Akram Osman. 2020. Recommendation system exploiting aspect-based opinion mining with deep learning method. Information Sciences 512 (2020), 1279--1292.
[8]
James Davidson, Benjamin Liebald, Junning Liu, Palash Nandy, Taylor Van Vleet, Ullas Gargi, Sujoy Gupta, Yu He, Mike Lambert, Blake Livingston, et al. 2010. The YouTube video recommendation system. In Proceedings of the fourth ACM conference on Recommender systems. 293--296.
[9]
Yashar Deldjoo, Mihai Gabriel Constantin, Bogdan Ionescu, Markus Schedl, and Paolo Cremonesi. 2018. MMTF-14K: a multifaceted movie trailer feature dataset for recommendation and retrieval. In Proceedings of the 9th ACM Multimedia Systems Conference. 450--455.
[10]
Yashar Deldjoo, Mehdi Elahi, Paolo Cremonesi, Franca Garzotto, Pietro Piazzolla, and Massimo Quadrana. 2016. Content-based video recommendation system based on stylistic visual features. Journal on Data Semantics 5, 2 (2016), 99--113.
[11]
Florian Eyben, Martin Wöllmer, and Björn Schuller. 2010. Opensmile: The Munich Versatile and Fast Open-Source Audio Feature Extractor. In Proceedings of the 18th ACM International Conference on Multimedia (MM '10). Association for Computing Machinery, New York, NY, USA, 1459--1462.
[12]
Yoav Goldberg and Omer Levy. 2014. word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method. arXiv preprint arXiv:1402.3722 (2014).
[13]
Andreas F Haas, Marine Guibert, Anja Foerschner, Sandi Calhoun, Emma George, Mark Hatay, Elizabeth Dinsdale, Stuart A Sandin, Jennifer E Smith, Mark JA Vermeij, et al. 2015. Can we measure beauty? Computational evaluation of coral reef aesthetics. PeerJ 3 (2015), e1390.
[14]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.
[15]
Xiangnan He, Hanwang Zhang, Min-Yen Kan, and Tat-Seng Chua. 2016. Fast matrix factorization for online recommendation with implicit feedback. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. 549--558.
[16]
Jonathan L Herlocker, Joseph A Konstan, Loren G Terveen, and John T Riedl. 2004. Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems (TOIS) 22, 1 (2004), 5--53.
[17]
Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).
[18]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2017. Imagenet classification with deep convolutional neural networks. Commun. ACM 60, 6 (2017), 84--90.
[19]
Sheng Li, Jaya Kawale, and Yun Fu. 2015. Deep collaborative filtering via marginalized denoising auto-encoder. In Proceedings of the 24th ACM international on conference on information and knowledge management. 811--820.
[20]
Blerina Lika, Kostas Kolomvatsos, and Stathes Hadjiefthymiades. 2014. Facing the cold start problem in recommender systems. Expert Systems with Applications 41, 4 (2014), 2065--2073.
[21]
Andriy Mnih and Russ R Salakhutdinov. 2007. Probabilistic matrix factorization. Advances in neural information processing systems 20 (2007).
[22]
Nour Nassar, Assef Jafar, and Yasser Rahhal. 2020. A novel deep multi-criteria collaborative filtering model for recommendation system. Knowledge-Based Systems 187 (2020), 104811.
[23]
Juan Ni, Zhenhua Huang, Yang Hu, and Chen Lin. 2022. A two-stage embedding model for recommendation with multimodal auxiliary information. Information Sciences 582 (2022), 22--37.
[24]
Sriram Pingali, Prabir Mondal, Daipayan Chakder, Sriparna Saha, and Angshuman Ghosh. 2022. Towards Developing a Multi-Modal Video Recommendation System. In International Joint Conference on Neural Networks, IJCNN 2022, Padua, Italy, July 18-23, 2022. IEEE, 1--8.
[25]
Zhongtai Qin and Mingjun Zhang. 2021. Towards a Personalized Movie Recommendation System: A Deep Learning Approach. In 2021 2nd International Conference on Artificial Intelligence and Information Systems. 1--5.
[26]
Tulika Saha, Aditya Prakash Patra, Sriparna Saha, and Pushpak Bhattacharyya. 2020. Towards Emotion-aided Multi-modal Dialogue Act Classification. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 4361--4372.
[27]
Tulika Saha, Apoorva Upadhyaya, Sriparna Saha, and Pushpak Bhattacharyya. 2021. Towards Sentiment and Emotion aided Multi-modal Speech Act Classification in Twitter. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021, Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tür, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (Eds.). Association for Computational Linguistics, 5727--5737.
[28]
Klaus Seyerlehner, Gerhard Widmer, Markus Schedl, and Peter Knees. 2010. Automatic music tag classification based on block-level. Proceedings of Sound and Music Computing (2010).
[29]
Stephen H Shum, Najim Dehak, Réda Dehak, and James R Glass. 2013. Unsupervised methods for speaker diarization: An integrated and iterative approach. IEEE Transactions on Audio, Speech, and Language Processing 21, 10 (2013), 2015--2028.
[30]
Florian Strub, Romaric Gaudel, and Jérémie Mary. 2016. Hybrid recommender system based on autoencoders. In Proceedings of the 1st workshop on deep learning for recommender systems. 11--16.
[31]
Robin Van Meteren and Maarten Van Someren. 2000. Using content-based filtering for recommendation. In Proceedings of the machine learning in the new information age: MLnet/ECML2000 workshop, Vol. 30. 47--56.
[32]
Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol, and Léon Bottou. 2010. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of machine learning research 11, 12 (2010).
[33]
Hao Wang, Naiyan Wang, and Dit-Yan Yeung. 2015. Collaborative deep learning for recommender systems. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. 1235--1244.
[34]
Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 28.
[35]
Triyanna Widiyaningtyas, Indriana Hidayah, and Teguh B Adji. 2021. User profile correlation-based similarity (UPCSim) algorithm in movie recommendation system. Journal of Big Data 8, 1 (2021), 1--21.
[36]
Hong-Jian Xue, Xinyu Dai, Jianbing Zhang, Shujian Huang, and Jiajun Chen. 2017. Deep matrix factorization models for recommender systems. In IJCAI, Vol. 17. Melbourne, Australia, 3203--3209.
[37]
Mohammad Hadi Valipour Zahra Zamanzadeh Darban. 2021. GHRS: Graph-based Hybrid Recommendation System with Application to Movie Recommendation.

Cited By

View all
  • (2024)Genre Effect Toward Developing a Multi-Modal Movie Recommendation System in Indian SettingIEEE Transactions on Consumer Electronics10.1109/TCE.2023.332400970:1(2517-2526)Online publication date: Feb-2024
  • (2024)Transformative Movie Discovery: Large Language Models for Recommendation and Genre PredictionIEEE Access10.1109/ACCESS.2024.348246112(186626-186638)Online publication date: 2024
  • (2023)An overview of video recommender systems: state-of-the-art and research issuesFrontiers in Big Data10.3389/fdata.2023.12816146Online publication date: 30-Oct-2023
  • Show More Cited By

Index Terms

  1. Graph Convolutional Neural Network for Multimodal Movie Recommendation
        Index terms have been assigned to the content through auto-classification.

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Conferences
        SAC '23: Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing
        March 2023
        1932 pages
        ISBN:9781450395175
        DOI:10.1145/3555776
        Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).

        Sponsors

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 07 June 2023

        Check for updates

        Author Tags

        1. multi-modal movie recommendation system
        2. graph convolutional neural network
        3. knowledge graph

        Qualifiers

        • Poster

        Conference

        SAC '23
        Sponsor:

        Acceptance Rates

        Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

        Upcoming Conference

        SAC '25
        The 40th ACM/SIGAPP Symposium on Applied Computing
        March 31 - April 4, 2025
        Catania , Italy

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)185
        • Downloads (Last 6 weeks)11
        Reflects downloads up to 16 Feb 2025

        Other Metrics

        Citations

        Cited By

        View all
        • (2024)Genre Effect Toward Developing a Multi-Modal Movie Recommendation System in Indian SettingIEEE Transactions on Consumer Electronics10.1109/TCE.2023.332400970:1(2517-2526)Online publication date: Feb-2024
        • (2024)Transformative Movie Discovery: Large Language Models for Recommendation and Genre PredictionIEEE Access10.1109/ACCESS.2024.348246112(186626-186638)Online publication date: 2024
        • (2023)An overview of video recommender systems: state-of-the-art and research issuesFrontiers in Big Data10.3389/fdata.2023.12816146Online publication date: 30-Oct-2023
        • (2023)A Multi-modal Multi-task based Approach for Movie Recommendation2023 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN54540.2023.10191882(1-8)Online publication date: 18-Jun-2023
        • (2023)Impulsion of Movie’s Content-Based Factors in Multi-modal Movie Recommendation SystemNeural Information Processing10.1007/978-981-99-8184-7_18(230-242)Online publication date: 26-Nov-2023

        View Options

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media