Skip to main content
Springer Nature Link
Log in

iMovieRec: a hybrid movie recommendation method based on a user-image-item model

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

We propose iMovieRec, a hybrid movie recommendation method that employs an image-user-item model, which utilizes both CF models and graph features. The purpose of this model is to efficiently learn the interactions between users and items and the key features of the poster images using single layer neural networks and matrix factorization. In particular, we consider various types of graph architectures to determine the graph structure that would express the relationship between users and items. The experimental results obtained using two benchmarking datasets indicate that iMovieRec is more efficient than the other recommendation models, which exhibit limited and varied image feature effects. In addition, we make both our datasets and the iMovieRec model publicly available.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. https://grouplens.org/datasets/movielens/

  2. https://www.themoviedb.org/

  3. The batch size of GraphRec was fixed at 1000.

  4. https://github.com/dxlabskku/iMovieRec

References

  1. Afoudi Y, Lazaar M, Al Achhab M (2021) Hybrid recommendation system combined content-based filtering and collaborative prediction using artificial neural network. Simul Model Pract Theory 113:102375

    Article  Google Scholar 

  2. Anwar T, Uma V (2021) Comparative study of recommender system approaches and movie recommendation using collaborative filtering. Int J Syst Assur Eng Manag 12(3):426–436

    Article  Google Scholar 

  3. Anwar T, Uma V, Hussain M, Pantula M et al (2022) Collaborative filtering and knn based recommendation to overcome cold start and sparsity issues: a comparative analysis. Multim Tools Appl 1–19

  4. Aqil M, Atasoy S, Kringelbach ML, Hindriks R (2021) Graph neural fields: a framework for spatiotemporal dynamical models on the human connectome. PLoS Comput Biol 17(1):e1008310

    Article  Google Scholar 

  5. Banerjee S, Jenamani M, Pratihar DK (2017) Properties of a projected network of a bipartite network. In: Proceedings of ICCSP ’17, IEEE, pp 0143–0147

  6. Bobadilla J, Bojorque R, Esteban AH, Hurtado R (2017) Recommender systems clustering using Bayesian non negative matrix factorization. IEEE Access 6:3549–3564

    Article  Google Scholar 

  7. Cevahir A (2017) Index partitioning through a bipartite graph model for faster similarity search in recommendation systems. Inf Syst Front 19(5):1161–1176

    Article  Google Scholar 

  8. Chen J, Zhao C, Chen L (2020) Collaborative filtering recommendation algorithm based on user correlation and evolutionary clustering. Complex Intell Syst 6:147–156

    Article  Google Scholar 

  9. Chen X, Yu Y, Jiang F, Zhang L, Gao R, Gao H (2020) Graph neural networks boosted personalized tag recommendation algorithm. In: Proceedings of IJCNN ’20, IEEE, pp 1–8

  10. Daneshvar H, Ravanmehr R (2022) A social hybrid recommendation system using lstm and cnn. Concurr Comput Pract Exp e7015

  11. Deng S, Huang L, Xu G, Wu X, Wu Z (2016) On deep learning for trust-aware recommendations in social networks. IEEE Trans Neural Netw Learn Syst 28(5):1164–1177

    Article  Google Scholar 

  12. Gan M, Cui H (2021) Exploring user movie interest space: a deep learning based dynamic recommendation model. Expert Syst Appl 173:114695

    Article  Google Scholar 

  13. Guan X, Li CT, Guan Y (2017) Matrix factorization with rating completion: an enhanced svd model for collaborative filtering recommender systems. IEEE Access 5:27668–27678

    Article  Google Scholar 

  14. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of CVPR ’16, pp 770–778

  15. He R, McAuley J (2016) Vbpr: visual Bayesian personalized ranking from implicit feedback. In: Proceedings of AAAI ’16, vol 30, pp 144–150

  16. Hwang TG, Park CS, Hong JH, Kim SK (2016) An algorithm for movie classification and recommendation using genre correlation. Multim Tools Appl 75(20):12843–12858

    Article  Google Scholar 

  17. Ivasic-Kos M, Pobar M, Mikec L (2014) Movie posters classification into genres based on low-level features. In: 2014 37th international convention on information and communication technology, electronics and microelectronics (MIPRO), IEEE, pp 1198–1203

  18. Jallouli M, Lajmi S, Amous I (2020) When contextual information meets recommender systems: extended svd++ models. Int J Comput Appl 1–8

  19. Jeong D, Oh S, Park E (2022) Demohash: hashtag recommendation based on user demographic information. Expert Syst Appl 210:118375

    Article  Google Scholar 

  20. Jiang J, Gao L, Yu S, Jin J, Yuan D (2017) Preferential attachment and the spreading influence of users in online social networks. In: Proceedings of of ICCC ’17, IEEE, pp 1–6

  21. Jiang S, Li K, Da Xu RY (2022) Magnitude bounded matrix factorisation for recommender systems. IEEE Trans Knowl Data Eng 34(4):1856–1869

    Article  Google Scholar 

  22. Kang G, Liu J, Cao B, Cao M (2020) Nafm: neural and attentional factorization machine for web api recommendation. In: Proceedings of ICWS ’20, IEEE, pp 330–337

  23. Kaushik A, Gupta S, Bhatia M (2018) A movie recommendation system using neural network. Int J Adv Res Ideas Innov Technol 11:425–430

    Google Scholar 

  24. Khaire UM, Dhanalakshmi R (2020) High-dimensional microarray dataset classification using an improved adam optimizer (iadam). J Ambient Intell Humaniz Comput 11(11):5187–5204

    Article  Google Scholar 

  25. Khelloufi A, Ning H, Dhelim S, Qiu T, Ma J, Huang R, Atzori L (2020) A social-relationships-based service recommendation system for siot devices. IEEE Internet Things J 8(3):1859–1870

    Article  Google Scholar 

  26. Kim D, Park C, Oh J, Yu H (2017) Deep hybrid recommender systems via exploiting document context and statistics of items. Inf Sci 417:72–87

    Article  Google Scholar 

  27. Kim D, Choi J, Ahn S, Park E (2021) A smart home dental care system: integration of deep learning, image sensors, and mobile controller. J Ambient Intell Humaniz Comput 1–9

  28. Kim J, Lee J, Park E, Han J (2020) A deep learning model for detecting mental illness from user content on social media. Sci Rep 10(1):1–6

    Google Scholar 

  29. Kim J, Ji H, Oh S, Hwang S, Park E, del Pobil AP (2021) A deep hybrid learning model for customer repurchase behavior. J Retail Consum Serv 59:102381

    Article  Google Scholar 

  30. Kumar S, De K, Roy PP (2020) Movie recommendation system using sentiment analysis from microblogging data. IEEE Trans Comput Soc Syst 7(4):915–923

    Article  Google Scholar 

  31. Kundalia K, Patel Y, Shah M (2020) Multi-label movie genre detection from a movie poster using knowledge transfer learning. Augment Hum Res 5:1–9

    Article  Google Scholar 

  32. Lee S, Ji H, Kim J, Park E (2021) What books will be your bestseller? a machine learning approach with amazon kindle. Electron Libr 39(1):137–151

    Article  Google Scholar 

  33. Lee S, Jeong D, Park E (2022) Multiemo: multi-task framework for emoji prediction. Knowl Based Syst 242:108437

    Article  Google Scholar 

  34. Liu JY (2018) A survey of deep learning approaches for recommendation systems. J Phys Conf Ser 1087(6):062022

    Article  Google Scholar 

  35. Liu ZY, Li SY, Chen S, Hu Y, Huang SJ (2020) Uncertainty aware graph gaussian process for semi-supervised learning. In: Proceedings of AAAI ’20, vol 34, pp 4957–4964

  36. Luo F, Ranzi G, Kong W, Liang G, Dong ZY (2020) Personalized residential energy usage recommendation system based on load monitoring and collaborative filtering. IEEE Trans Ind Inform 17(2):1253–1262

    Article  Google Scholar 

  37. Mazumdar P, Patra BK, Babu KS (2020) Cold-start point-of-interest recommendation through crowdsourcing. ACM Trans Web 14(4):1–36

    Article  Google Scholar 

  38. Moradabadi B, Meybodi MR (2018) Link prediction in weighted social networks using learning automata. Eng Appl Artif Intell 70:16–24

    Article  MATH  Google Scholar 

  39. Oh S, Ji H, Kim J, Park E, del Pobil AP (2022) Deep learning model based on expectation-confirmation theory to predict customer satisfaction in hospitality service. Inf Technol Tour 24(1):109–126

    Article  Google Scholar 

  40. Pan Y, He F, Yu H (2019) A novel enhanced collaborative autoencoder with knowledge distillation for top-n recommender systems. Neurocomputing 332:137–148

    Article  Google Scholar 

  41. Pan Y, He F, Yu H (2020) A correlative denoising autoencoder to model social influence for top-n recommender system. Front Comput Sci 14(3):1–13

    Article  Google Scholar 

  42. Pan Y, He F, Yu H (2020) Learning social representations with deep autoencoder for recommender system. World Wide Web 23(4):2259–2279

    Article  Google Scholar 

  43. Park E (2023) Crnet: a multimodal deep convolutional neural network for customer revisit prediction. J Big Data 10(1):1–10

    Article  Google Scholar 

  44. Pirasteh P, Bouguelia MR, Santosh K (2021) Personalized recommendation: an enhanced hybrid collaborative filtering. Adv Comput Intell 1(4):1–8

    Article  Google Scholar 

  45. Ranjan AA, Rai A, Haque S, Lohani BP, Kushwaha PK (2019) An approach for netflix recommendation system using singular value decomposition. J Comput Math Sci 10(4):774–779

    Google Scholar 

  46. Rashed A, Grabocka J, Schmidt-Thieme L (2019) Attribute-aware non-linear co-embeddings of graph features. In: Proceedings of Recsys ’19, pp 314–321

  47. Reddy S, Nalluri S, Kunisetti S, Ashok S, Venkatesh B (2019) Content-based movie recommendation system using genre correlation. In: Smart intelligent computing and applications. Springer, pp 391–397

  48. Salehani YE, Arabnejad E, Rahiche A, Bakhta A, Cheriet M (2020) Msdb-nmf: multispectral document image binarization framework via non-negative matrix factorization approach. IEEE Trans Image Process 29:9099–9112

    Article  MathSciNet  MATH  Google Scholar 

  49. Shah M, Parikh D, Deshpande B (2016) Movie recommendation system employing latent graph features in extremely randomized trees. In: Proceedings of ICTCS ’16, pp 1–4

  50. Singh M (2020) Scalability and sparsity issues in recommender datasets: a survey. Knowl Inf Syst 62(1):1–43

    Article  Google Scholar 

  51. Singh PK, Pramanik PKD, Dey AK, Choudhury P (2021) Recommender systems: an overview, research trends, and future directions. Int J Bus Syst Res 15(1):14–52

    Article  Google Scholar 

  52. Son J, Kim SB (2017) Content-based filtering for recommendation systems using multiattribute networks. Expert Syst Appl 89:404–412

    Article  Google Scholar 

  53. Tahmasebi F, Meghdadi M, Ahmadian S, Valiallahi K (2021) A hybrid recommendation system based on profile expansion technique to alleviate cold start problem. Multim Tools Appl 80(2):2339–2354

    Article  Google Scholar 

  54. Tepper M, Sapiro G (2017) Nonnegative matrix underapproximation for robust multiple model fitting. In: Proceedings of CVPR ’17, pp 2059–2067

  55. Vilakone P, Park DS, Xinchang K, Hao F (2018) An efficient movie recommendation algorithm based on improved k-clique. Hum Centr Comput Inf Sci 8(1):38

    Article  Google Scholar 

  56. Wang D, Liang Y, Xu D, Feng X, Guan R (2018) A content-based recommender system for computer science publications. Knowl Based Syst 157:1–9

    Article  Google Scholar 

  57. Wang F, Zhu H, Srivastava G, Li S, Khosravi MR, Qi L (2021) Robust collaborative filtering recommendation with user-item-trust records. IEEE Trans Comput Soc Syst 9(4):986–996

    Article  Google Scholar 

  58. Wang J, Zhang X, Chen L (2021) How well do pre-trained contextual language representations recommend labels for github issues? Knowl Based Syst 107476

  59. Wang X, He X, Wang M, Feng F, Chua TS (2019) Neural graph collaborative filtering. In: Proceedings of SIGIR ’19, pp 165–174

  60. Wang X, Jin H, Zhang A, He X, Xu T, Chua TS (2020) Disentangled graph collaborative filtering. In: Proceedings of SIGIR ’20, pp 1001–1010

  61. Wang Z, Chen C, Li W (2017) Predictive network representation learning for link prediction. In: Proceedings of SIGIR ’17, pp 969–972

  62. Wei J, He J, Chen K, Zhou Y, Tang Z (2017) Collaborative filtering and deep learning based recommendation system for cold start items. Expert Syst Appl 69:29–39

    Article  Google Scholar 

  63. Xu C, Wu M (2020) Learning feature interactions with lorentzian factorization machine. In: Proceedings of AAAI ’20, vol 34, pp 6470–6477

  64. Yang Q, Li H, Zhou Y, Zhang J, Fuentes S (2020) Hybrid cf on modeling feature importance with joint denoising autoencoder and svd++. In: Proceedings of CollaborateCom ’20. Springer, pp 290–304

  65. Yu H, Park E (2023) A harmless webtoon for all: an automatic age-restriction prediction system for webtoon contents. Telem Inform 76:101906

    Article  Google Scholar 

  66. Yu W, Qin Z (2020) Graph convolutional network for recommendation with low-pass collaborative filters. In: Proceedings of ICML ’20, PMLR, pp 10936–10945

  67. Zhang J, Yang Y, Zhuo L, Tian Q, Liang X (2019) Personalized recommendation of social images by constructing a user interest tree with deep features and tag trees. IEEE Trans Multim 21(11):2762–2775

    Article  Google Scholar 

  68. Zhang L, Li J, Zhang Q, Meng F, Teng W (2019) Domain knowledge-based link prediction in customer-product bipartite graph for product recommendation. Int J Inf Technol Decis Mak 18(01):311–338

    Article  Google Scholar 

  69. Zhang S, Yao L, Xu X (2017) Autosvd++ an efficient hybrid collaborative filtering model via contractive auto-encoders. In: Proceedings of SIGIR ’17, pp 957–960

  70. Zhao L, Lu Z, Pan SJ, Yang Q (2016) Matrix factorization+ for movie recommendation. In: Proceedings of IJCAI ’16, pp 3945–3951

  71. Zhao Z, Yang Q, Lu H, Weninger T, Cai D, He X, Zhuang Y (2017) Social-aware movie recommendation via multimodal network learning. IEEE Trans Multim 20(2):430–440

    Article  Google Scholar 

  72. Zheng L, Lu CT, Jiang F, Zhang J, Yu PS (2018) Spectral collaborative filtering. In: Proceedings of Recsys ’18, pp 311–319

  73. Zitouni H, Meshoul S, Taouche K (2017) Improving content based recommender systems using linked data cloud and foaf vocabulary. In: Proceedings of WI ’17, pp 988–992

  74. Zuo T, Zhu S, Lu J (2020) A hybrid recommender system combing singular value decomposition and linear mixed model. In: Science and information conference. Springer, pp 347–362

Download references

Funding

This work was supported by the Institute of Information and Communications Technology Planning and Evaluation (IITP) Grant funded by the Korean government (MSIT) (no. 2021-0-00358, AI-Big data-based Cyber Security Orchestration and Automated Response Technology Development). This research was also supported by the MSIT, Korea, under the ICAN program (no. IITP-2020-0-01816) supervised by the IITP.

Author information

Authors and Affiliations

  1. Department of Interaction Science, Sungkyunkwan University, Seoul, Republic of Korea

    Syjung Hwang

  2. Dreamus Company, Seoul, Republic of Korea

    Syjung Hwang

  3. Department of Applied Artificial Intelligence, Sungkyunkwan University, Seoul, Republic of Korea

    Hyeongjin Ahn

  4. University Jaume-I, Castellon, Spain

    Hyeongjin Ahn

  5. Department of Interaction Science, Sungkyunkwan University, Seoul, Republic of Korea

    Eunil Park

  6. Department of Applied Artificial Intelligence, Sungkyunkwan University, Seoul, Republic of Korea

    Eunil Park

Authors
  1. Syjung Hwang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Hyeongjin Ahn
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Eunil Park
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Eunil Park.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest.

Ethical approval

This research was conducted with IRB approval: Sungkyunkwan University IRB (2020-11-013).

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hwang, S., Ahn, H. & Park, E. iMovieRec: a hybrid movie recommendation method based on a user-image-item model. Int. J. Mach. Learn. & Cyber. 14, 3205–3216 (2023). https://doi.org/10.1007/s13042-023-01828-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-023-01828-3

Keywords