Skip to main content
SpringerLink
Log in

A hybrid recommendation system considering visual information for predicting favorite restaurants

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Restaurant recommendation is one of the most interesting recommendation problems because of its high practicality and rich context. Many works have been proposed to recommend restaurants by considering user preference, restaurant attributes, and socio-demographic behaviors. In addition to these, many customers review restaurants in blog articles where text-based subjective comments and various photos may be available. In this paper, we especially investigate the influence of visual information, i.e., photos taken by customers and put on blogs, on predicting favorite restaurants for any given user. By considering visual information as the intermediate, we will integrate two common recommendation approaches, i.e., content-based filtering and collaborative filtering, and show the effectiveness of considering visual information. More particularly, we advocate that, in addition to text information or metadata, restaurant attributes and user preference can both be represented by visual features. Heterogeneous items can thus be modeled in the same space, and thus two types of recommendation approaches can be linked. Through experiments with various settings, we verify that visual information effectively aids favorite restaurant prediction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12

Similar content being viewed by others

Notes

  1. http://hungry.9ifriend.com/main/

References

  1. Adomavicius, G., Tuzhilin, A.: Towards the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng. 17(6), 734–749 (2005)

    Article  Google Scholar 

  2. Berlin, B., Kay, P.: Basic Color Terms: Their Universality and Evolution. University of California Press (1991)

  3. Bostandjiev, S., O’Donovan, J., Hollerer, T.: Tasteweights: A visual interactive hybrid recommender system. In: Proceedings of ACM Conference on Recommender Systems, pp. 361–364 (2010)

  4. Breese, J., Heckerman, D., Kadie, C.: Empirical analysis of predictive algorithms for collaborative filtering. In: Proceedings of Conference on Uncertainty in Articial Intelligence, pp. 43–52 (1998)

  5. Burke, R.: Hybrid recommender systems: Survey and experiments. User Model. User-Adapt. Inter. 12(4), 331–370 (2002)

    Article  MATH  Google Scholar 

  6. Chang, C.C., Lin, C.J.: Libsvm: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 27 (2011)

    Article  Google Scholar 

  7. Chu, C.H., Wu, S.H.: A chinese restaurant recommendation system based on mobile context-aware services. In: Proceedings of IEEE International Conference on Mobile Data Management, pp. 116–118 (2013)

  8. Chu, W.T., Huang, W.H.: Cultural difference and visual information on hotel rating prediction. WWW J. Inter. Web Inf. Syst. (2016)

  9. Cordeiro, F., Bales, E., Cherry, E., Fogarty, J.: Rethinking the mobile food journal: Exploring opportunities for lightweight photo-based capture. In: Proceedings of ACM Conference on Human Factors in Computing Systems, pp. 3207–3216 (2015)

  10. Fu, Y., Liu, B., Ge, Y., Yao, Z., Xiong, H.: User preference learning with multiple information fusion for restaurant recommendation. In: Proceedings of SIAM International Conference on Data Mining, pp. 470–478 (2014)

  11. Gao, Y., Yu, W., Chao, P., Zhang, R., Zhou, A., Yang, X.: A restaurant recommendation system by analyzing ratings and aspects in reviews. In: Database Systems for Advanced Applications, pp. 526–530 (2015)

  12. Gupta, A., Singh, K.: Location based personalized restaurant recommendation system for mobile environments. In: Proceedings of International Conference on Advances in Computing, Communications and Informatics (2013)

  13. Khan, F., Anwer, R., van de Weijer, J., Bagdanov, A., Vanrell, M., Lopez, A.: Color attributes for object detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3306–3313 (2012)

  14. Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009)

    Article  Google Scholar 

  15. Krizhevsky, A., Sutskever, I., Hinton, G.: Imagenet classification with deep convolutional neural networks. In: Proceedings of Advances in Neural Information Processing Systems, pp. 1106–1114 (2012)

  16. Kuo, W.T., Wang, Y.C., Tsai, R.T.H., Hsu, J.Y.J.: Contextual restaurant recommendation utilizing implicit feedback. In: Proceedings of Wireless and Optical Communication Conference, pp. 170–174 (2015)

  17. Linden, G., Smith, B., York, J.: Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Int. Comput. 7(1), 76–80 (2003)

    Article  Google Scholar 

  18. Liu, X., Aggarwal, C., Li, Y.F., Kong, X., Sun, X., Sathe, S.: Kernelized matrix factorization for collaborative filtering. In: Proceedings of SIAM International Conference on Data Mining (2016)

  19. Lops, P, de Gemmis, M., Semeraro, G.: Content-based recommender systems: State of the art and trends, pp 73–105. Recommender Systems Handbook (2011)

  20. Musto, C.: Enhanced vector space models for content-based recommender systems. In: Proceedings of ACM Conference on Recommender Systems, pp. 361–364 (2010)

  21. Pazzani, M., Billsus, D.: Content-based recommendation systems. Adapt. Web, 325–341 (2007)

  22. Rendle, S.: Factorization machines. In: Proceedings of IEEE International Conference on Data Mining, pp. 995–1000 (2010)

  23. Rendle, S., Freudenthaler, C., Gantner, Z., Schmidt-Thieme, L.: Bpr: Bayesian personalized ranking from implicit feedback. In: Proceedings of the Conference on Uncertainty in Artificial Intelligence, pp. 452–461 (2009)

  24. Sarwar, B., Karypis, G., Konstan, J., Riedl, J.: Application of dimensionality reduction in recommender system: A case study. In: Proceedings of ACM WebKDD Workshop (2000)

  25. Shih, Y.Y., Liu, D.R.: Hybrid recommendation approaches: Collaborative filtering via valuable content information. In: Proceedings of Hawaii International Conference on System Sciences (2005)

  26. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of International Conference on Learning Representations (2015)

  27. Strub, F., Mary, J., Gaudel, R.: Hybrid recommender system based on autoencoders. (2016). arXiv:1606.07659

  28. Su, X., Khoshgoftaar, T.: A survey of collaborative filtering techniques. Advances in Artificial Intelligence 2009 (2009)

  29. Sun, J., Xiong, Y., Zhu, Y., Liu, J., Guan, C., Xiong, H.: Multi-source information fusion for personalized restaurant recommendation. In: Proceedings of International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 983–986 (2015)

  30. van den Oord, A., Dieleman, S., Schrauwen, B.: Deep content-based music recommendation. In: Proceedings of Advances in Neural Information Processing Systems (2013)

  31. Vedaldi, A., Lenc, K.: Matconvnet: Convolutional neural networks for matlab. In: Proceedings of ACM International Conference on Multimedia, pp. 689–692 (2015)

  32. Wang, Y., Stash, N., Aroyo, L., Hollink, L., Schreiber, G.: Semantic relations for content-based recommendations. In: Proceedings of International Conference on Knowledge Capture, pp. 209–210 (2010)

  33. Wang, Z., Liao, J., Cao, Q., Qi, H., Wang, Z.: Friendbook: A semantic-based friend recommendation system for social networks. IEEE Trans. Mob. Comput. 14(3), 538–551 (2015)

    Article  Google Scholar 

  34. Yu, K., Zhu, S., Lafferty, J., Gong, Y.: Fast nonparametric matrix factorization for large-scale collaborative filtering. In: Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 211–218 (2009)

  35. Zhang, F., Zheng, K., Yuan, N.J., Xie, X., Chen, E., Zhou, X.: A novelty-seeking based dining recommender system (2015)

  36. Zheng, L., Wang, S., Tian, Q.: Coupled binary embedding for large-scale image retrieval. IEEE Trans. Image Process. 23(8), 3368–3380 (2014)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

The work was partially supported by the Ministry of Science and Technology in Taiwan under the grant MOST103-2221-E-194-027-MY3, MOST104-2221-E-194-014, and MOST105-2628-E-194-001-MY2.

Author information

Authors and Affiliations

  1. National Chung Cheng University, Chiayi County, Taiwan

    Wei-Ta Chu & Ya-Lun Tsai

Authors
  1. Wei-Ta Chu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ya-Lun Tsai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei-Ta Chu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chu, WT., Tsai, YL. A hybrid recommendation system considering visual information for predicting favorite restaurants. World Wide Web 20, 1313–1331 (2017). https://doi.org/10.1007/s11280-017-0437-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-017-0437-1

Keywords

Search

Navigation