Skip to main content
SpringerLink
Log in

GMiRec: A Multi-image Visual Recommendation Model Based on a Gated Neural Network

  • Conference paper
  • First Online:
Knowledge Science, Engineering and Management (KSEM 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14118))

  • 516 Accesses

Abstract

Making use of visual perception in recommender systems is becoming more and more important. In the existing visual recommendation models (VRMs), the visual features of items are usually extracted based on the pre-trained convolutional neural network, and then combined with the non-visual features modeling to complete the prediction of users’ interest. There are two challenges in this field so far. First, most VRMs are developed around single-image items, and how to more effectively mine the visual features of multi-image items is seldom considered. Second, most models do not consider the distribution difference between the training datasets of the pre-trained model and the datasets for recommendation when extracting visual features based on the pre-training model, which may deepen the gap in the convolutional neural network’s understanding of image semantics on datasets. To address the above challenges, a Multi-image Visual Recommendation Model based on a Gated Neural Network (GMiRec) is proposed. It performs different forms of pooling operations on the visual features of multi-image items and uses the feed-forward neural network to realize the fusion of the multi-image visual information. In addition, a gated neural network taking item categories as input is designed to achieve supervised dimensionality reduction on the item visual features, which alleviates the problem of semantic gap. Experiments conducted on the Amazon datasets show that the proposed model is significantly improved compared with the existing models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Zhang, S., Yao, L., Sun, A., et al.: Deep learning based recommender system: a survey and new perspectives. ACM Comput. Surv. 52(1), 1–38 (2019)

    Article  Google Scholar 

  2. Linden, G., Smith, B., York, J.: Item-to-item collaborative filtering. IEEE Internet Comput. 7(1), 76–80 (2003)

    Article  Google Scholar 

  3. Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009)

    Article  Google Scholar 

  4. Koren, Y.: Factor in the neighbors: scalable and accurate collaborative filtering. ACM Trans. Knowl. Discov. Data 4(1), 1–24 (2010)

    Article  MathSciNet  Google Scholar 

  5. Pan, R., Zhou, Y., Cao, B., et al.: One-class collaborative filtering. In: 8th IEEE International Conference on Data Mining, pp. 502–511. IEEE, Piscataway (2008)

    Google Scholar 

  6. He, X., Liao, L., Zhang, H., et al.: Neural collaborative filtering. In: 26th International Conference on World Wide Web Companion, pp.173–182. ACM, New York (2017)

    Google Scholar 

  7. He, X., Du, X., Wang, X., et al.: Outer product-based neural collaborative filtering. In: 27th International Joint Conference on Artificial Intelligence, pp. 2227–2233. ACM, New York (2018)

    Google Scholar 

  8. Truong, Q., Salah, A., Lauw, H.: Multi-modal recommender systems: hands-on exploration. In: 15th ACM Conference on Recommender Systems, pp. 834–837. ACM, New York (2021)

    Google Scholar 

  9. Liu, H., Lu, J., Yang, H., et al.: Category-specific CNN for visual-aware CTR prediction at JD. Com. In: 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2686–2696. ACM, New York (2020)

    Google Scholar 

  10. Krizhevsky, A., Sutskever, I., Hinton, G.: Imagenet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017)

    Article  Google Scholar 

  11. He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: 13th IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778. IEEE, Piscataway (2016)

    Google Scholar 

  12. Mo, K., Liu, B., Xiao, L., Li, Y., Jiang, J.: Image feature learning for cold start problem in display advertising. In: 24th International Conference on Artificial Intelligence, pp. 3728–3734. IJCAI, Buenos Aires Argentina (2015)

    Google Scholar 

  13. Zhao, Z., Li, L., Zhang, B., et al.: What you look matters? Offline evaluation of advertising creatives for cold-start problem. In: 28th ACM International Conference on Information and Knowledge Management, pp. 2605–2613. ACM, New York (2019)

    Google Scholar 

  14. He, R., Mcauley, J.: VBPR: visual Bayesian personalized ranking from implicit feed-back. In: 16th AAAI Conference on Artificial Intelligence, pp. 144–150. AAAI, Menlo Park (2016)

    Google Scholar 

  15. Liu, Q., Wu, S., Wang, L.: Deepstyle: learning user preferences for visual recommendation. In: 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 841–844. ACM, New York (2017)

    Google Scholar 

  16. He, R., Mcauley, J.: Ups and downs: modeling the visual evolution of fashion trends with one-class collaborative filtering. In: 25th International Conference on World Wide Web, pp. 507–517. WWW, Switzerland (2016)

    Google Scholar 

  17. Yu, W., Zhang, H., He, X., et al.: Aesthetic-based clothing recommendation. In: 18th World Wide Web Conference, pp. 649–658. WWW, Switzerland (2018)

    Google Scholar 

  18. Yu, W., et al.: Visually aware recommendation with aesthetic features. VLDB J. 30(4), 495–513 (2021). https://doi.org/10.1007/s00778-021-00651-y

    Article  Google Scholar 

  19. Zhang, F., Yuan, N., Lian, D., et al.: Collaborative knowledge base embedding for recommender systems. In: 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 353–362. ACM, New York (2016)

    Google Scholar 

  20. Chen, X., Zhao, P., Liu, Y., et al.: Exploiting visual contents in posters and still frames for movie recommendation. IEEE Access 6, 68874–68881 (2018)

    Article  Google Scholar 

  21. Wu, C., Wu, F., Qi, T., et al.: MM-Rec: visiolinguistic model empowered multimodal news recommendation. In: 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2560–2564. ACM, New York (2022)

    Google Scholar 

  22. Lei, C., Liu, D., Li, W., et al.: Comparative deep learning of hybrid representations for image recommendations. In: 13th Conference on Computer Vision and Pattern Recognition, pp. 2545–2553. IEEE, Piscataway (2016)

    Google Scholar 

  23. Yilma, B., Leiva, L.: CuratorNet: visually-aware recommendation of art images. In: 23th Conference on Human Factors in Computing Systems, pp. 1–17. ACM, New York (2023)

    Google Scholar 

  24. Sandler, M., Howard, A., Zhu, M., et al.: MobileNetV2: inverted residuals and linear bottlenecks. In: 15th IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–1520. IEEE, Piscataway (2018)

    Google Scholar 

  25. Boureau, Y., Ponce, J., LeCun, Y.: A theoretical analysis of feature pooling in visual recognition, pp. 111–118. Omnipress, United States (2010)

    Google Scholar 

  26. Hein, A.: Identification and bridging of semantic gaps in the context of multi-domain engineering. In: 2010 Forum on Philosophy, Engineering & Technology, pp. 57–58 (2010)

    Google Scholar 

  27. Rendle, S., Feudenthaler, C., Ganther, Z., et al.: BPR: Bayesian personalized ranking from implicit feedback. In: 25th Conference on Uncertainty in Artificial Intelligence, pp. 452–461. ACM, New York (2009)

    Google Scholar 

  28. Ni, J., Li, J., McAuley, J.: Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pp. 188–197. ACL, Stroudsburg (2019)

    Google Scholar 

  29. Jonathan, O.: Product recommendation based on visual similarity. https://www.kaggle.com/code/jonathanoheix/product-recommendation-based-on-visual-similarity. Accessed 1 Feb 2023

  30. Niu, W., Ccaverlee, J., Lu, H.: Neural personalized ranking for image recommendation. In: 8th ACM International Conference on Web Search and Data Mining. ACM, New York (2018)

    Google Scholar 

  31. Microsoft. Neural Network Intelligence. http://github.com/Microsoft/nni. Accessed 1 Feb 2023

  32. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

    Google Scholar 

Download references

Acknowledgement

This work was supported by the National Natural Science Foundation of China (Nos. 62077038, 61672405, 62176196 and 62271374).

Author information

Authors and Affiliations

  1. Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education, Collaborative Innovation Center of Quantum Information of Shaanxi Province, School of Artificial Intelligence, Xidian University, Xi’an, 710071, China

    Caihong Mu, Xin Tang & Jiashen Luo

  2. School of Electronic Engineering, Xidian University, Xi’an, 710071, China

    Yi Liu

Authors
  1. Caihong Mu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xin Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiashen Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiashen Luo .

Editor information

Editors and Affiliations

  1. Peking University, Beijing, China

    Zhi Jin

  2. South China Normal University, Guangzhou, China

    Yuncheng Jiang

  3. Babeș-Bolyai University, Cluj-Napoca, Romania

    Robert Andrei Buchmann

  4. Ulster University, Belfast, UK

    Yaxin Bi

  5. Babeș-Bolyai University, Cluj-Napoca, Romania

    Ana-Maria Ghiran

  6. South China Normal University, Guangzhou, China

    Wenjun Ma

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mu, C., Tang, X., Luo, J., Liu, Y. (2023). GMiRec: A Multi-image Visual Recommendation Model Based on a Gated Neural Network. In: Jin, Z., Jiang, Y., Buchmann, R.A., Bi, Y., Ghiran, AM., Ma, W. (eds) Knowledge Science, Engineering and Management. KSEM 2023. Lecture Notes in Computer Science(), vol 14118. Springer, Cham. https://doi.org/10.1007/978-3-031-40286-9_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-40286-9_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-40285-2

  • Online ISBN: 978-3-031-40286-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation