Skip to main content
SpringerLink
Log in

Efficient processing of recommendation algorithms on a single-machine-based graph engine

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The wide use of recommendation systems includes more users and items in system operations, leading to a significant increase in the size of related datasets. However, recommendation algorithms on existing single-machine-based graph engines have been developed without considering the important characteristics of recommendation datasets, i.e., huge size and power-law degree distribution. In this paper, we address how to realize efficient graph- and matrix-factorization-based recommendation algorithms, handling recommendation datasets on RealGraph, a state-of-the-art single-machine-based graph engine. Through extensive experiments, we demonstrate that our recommendation algorithms on RealGraph universally and consistently outperform the algorithms on other graph engines over all datasets up to 34 times.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. http://grouplens.org/datasets/movielens/1m/.

  2. https://archive.org/download/nf_prize_dataset.tar.

References

  1. Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans Knowl Data Eng 17(6):734–749

    Article  Google Scholar 

  2. Carey MJ, DeWitt DJ, Franklin MJ, Hall NE, McAuliffe ML, Naughton JF, Schuh DT, Solomon MH, Tan C, Tsatalos OG, White SJ, Zwilling, MJ (1994) Shoring up persistent applications. In: Proceedings of the ACM International Conference on Management of Data, ACM, pp 383–394

  3. Carey MJ, DeWitt DJ, Richardson JE, Shekita EJ (1986) Object and file management in the EXODUS extensible database system. University of Wisconsin-Madison, Madison

  4. Chen H, Li Z, Hu W (2016) An improved collaborative recommendation algorithm based on optimized user similarity. J Supercomput 72(7):2565–2578

    Article  Google Scholar 

  5. Chen, HC., Chen, AL (2001) A music recommendation system based on music data grouping and user interests. In: Proceedings of the International Conference on Information and Knowledge Management, ACM, pp 231–238

  6. Chi Y, Dai G, Wang Y, Sun G, Li G, Yang H (2016) Nxgraph: an efficient graph processing system on a single machine. In: Proceedings of the IEEE International Conference on Data Engineering, pp 409–420

  7. Chou HT, Dewitt DJ, Katz RH, Klug AC (1985) Design and implementation of the wisconsin storage system. Softw Pract Exp 15(10):943–962

    Article  Google Scholar 

  8. Da Zheng DM, Burns R, Vogelstein J, Priebe CE, Szalay AS (2015) Flashgraph: processing billion-node graphs on an array of commodity ssds. In: Proceedings of the USENIX Conference on File and Storage Technologies, pp 45–58

  9. Fousek J (2017) Efficient sparse matrix-delayed vector multiplication for discretized neural field model. J Supercomput 74(5):1–22

    Google Scholar 

  10. Gomez-Uribe CA, Hunt N (2016) The netflix recommender system: algorithms, business value, and innovation. ACM Trans Manag Inf Syst (TMIS) 6(4):13:1–13:19

    Google Scholar 

  11. Gonzalez JE, Low Y, Gu H, Bickson D, Guestrin C (2012) Powergraph: distributed graph-parallel computation on natural graphs. In: Proceedings of the USENIX Symposium on Operating Systems Design and Implementation, vol 12, p 2

  12. Ha J, Kwon SH, Kim SW, Faloutsos C, Park S (2012) Top-n recommendation through belief propagation. In: Proceedings of the ACM International Conference on Information and Knowledge Management, ACM, pp 2343–2346

  13. Han WS, Lee S, Park K, Lee JH, Kim MS, Kim J, Yu H (2013) Turbograph: a fast parallel graph engine handling billion-scale graphs in a single pc. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, pp 77–85

  14. Hung L (2005) A personalized recommendation system based on product taxonomy for one-to-one marketing online. Expert Syst Appl 29(2):383–392

    Article  Google Scholar 

  15. Hwang WS, Li S, Kim SW, Lee K (2014) Data imputation using a trust network for recommendation. In: Proceedings of the International Conference on World Wide Web, pp 299–300

  16. Jamali M, Ester M (2010) A matrix factorization technique with trust propagation for recommendation in social networks. In: Proceedings of the ACM Conference on Recommender Systems, ACM, pp 135–142

  17. Jang MH, Faloutsos C, Kim SW, Kang U, Ha J (2016) Pin-trust: fast trust propagation exploiting positive, implicit, and negative information. In: Proceedings of the ACM International on Conference on Information and Knowledge Management, ACM, pp 629–638

  18. Jo YY, Jang MH, Jung H, Kim, SW (2018) A high-performance graph engine for efficient social network analysis. In: Proceedings of the International Conference on World Wide Web, pp 61–62

  19. Kim J, Kang S, Lim Y, Kim HM (2013) Recommendation algorithm of the app store by using semantic relations between apps. J Supercomput 65(1):16–26

    Article  Google Scholar 

  20. Kim MS, An K, Park H, Seo H, Kim J (2016) GTS: a fast and scalable graph processing method based on streaming topology to GPUS. In: Proceedings of the ACM International Conference on Management of Data, ACM, pp 447–461

  21. Koren Y, Bell R, Volinsky C (2009) Matrix factorization techniques for recommender systems. Computer 42(8):30–37

    Article  Google Scholar 

  22. Kyrola A, Blelloch GE, Guestrin C (2012) Graphchi: large-scale graph computation on just a pc. In: Proceedings of the USENIX Symposium on Operating Systems Design and Implementation, USENIX, pp 31–46

  23. Lee YC, Kim SW, Lee D (2018) GOCCF: graph-theoretic one-class collaborative filtering based on uninteresting items. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 3448–3456

  24. Leskovec J, Kleinberg J, Faloutsos C (2007) Graph evolution: densification and shrinking diameters. ACM Trans Knowl Discov Data 1(1):2

    Article  Google Scholar 

  25. Liu NN, Yang Q (2008) Eigenrank: a ranking-oriented approach to collaborative filtering. In: Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM, pp 83–90

  26. Ma L, Yang Z , Chen H , Xue J, Dai Y (2017) Garaph: efficient GPU-accelerated graph processing on a single machine with balanced replication. In: Proceedings of the USENIX Annual Technical Conference, pp 195–207

  27. Maass S, Min C, Kashyap S, Kang W, Kumar M, Kim T (2017) Mosaic: Processing a trillion-edge graph on a single machine. In: Proceedings of the European Conference on Computer Systems, pp 527–543

  28. Nelakurthi, AR, He, J (2017) Finding cut from the same cloth: cross network link recommendation via joint matrix factorization. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 1467–1473

  29. Paterek A (2007) Improving regularized singular value decomposition for collaborative filtering. In: Proceedings of the KDD Cup and Workshop 2007, pp 5–8

  30. Roy A, Mihailovic I, Zwaenepoel W (2013) X-stream: edge-centric graph processing using streaming partitions. In: Proceedings of the ACM Symposium on Operating Systems Principles, ACM, pp 472–488

  31. Schafer JB, Frankowski D, Herlocker J, Sen S (2007) Collaborative filtering recommender systems. In: The Adaptive Web, vol 4321. Springer, Berlin, pp 291–324

  32. Shi L, Zhao WX, Shen YD (2017) Local representative-based matrix factorization for cold-start recommendation. ACM Trans Inf Syst 36(2):22

    Article  Google Scholar 

  33. Takács G, Pilászy I, Németh B, Tikk D (2008) Matrix factorization and neighbor based algorithms for the netflix prize problem. In: Proceedings of the ACM Conference on Recommender Systems, ACM, pp 267–274

  34. Takács G, Tikk D (2012) Alternating least squares for personalized ranking. In: Proceedings of the ACM Conference on Recommender Systems, ACM, pp 83–90

  35. Walter FE, Battiston S, Schweitzer F (2008) A model of a trust-based recommendation system on a social network. Auton Agent Multi-Agent Syst 16(1):57–74

    Article  Google Scholar 

  36. Yang Z, Chen W, Huang J (2017) Enhancing recommendation on extremely sparse data with blocks-coupled non-negative matrix factorization. Neurocomputing 278:126–133

    Article  Google Scholar 

  37. Yildirim H, Krishnamoorthy MS (2008) A random walk method for alleviating the sparsity problem in collaborative filtering. In: Proceedings of the ACM Conference on Recommender Systems, ACM, pp 131–138

  38. Zhou Y, Wilkinson D, Schreiber R, Pan R (2008) Large-scale parallel collaborative filtering for the netflix prize. In: Proceedings of the International Conference on Algorithmic Applications in Management, Springer, Berlin, pp 337–348

  39. Zhu X, Han W, Chen W (2015) Gridgraph: large-scale graph processing on a single machine using 2-level hierarchical partitioning. In: Proceedings of the USENIX Annual Technical Conference, USENIX, pp 375–386

Download references

Acknowledgements

This research was supported by (1) Next-Generation Information Computing Development Program through the National Research Foundation of Korea (NRF) Funded by the Ministry of Science, ICT (MSIT) (NRF-2017M3C4A7069440), (2) NRF Funded by MSIT (NRF-2017R1A2B3004581), and (3) Semiconductor Industry Collaborative Project between Hanyang University and Samsung Electronics Co. Ltd.

Author information

Authors and Affiliations

  1. Department of Computer and Software, Hanyang University, Seoul, Korea

    Yong-Yeon Jo, Myung-Hwan Jang & Sang-Wook Kim

  2. Department of Software and Computer Engineering, Ajou University, Suwon, Korea

    Kyungsik Han

Authors
  1. Yong-Yeon Jo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Myung-Hwan Jang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sang-Wook Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kyungsik Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kyungsik Han.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jo, YY., Jang, MH., Kim, SW. et al. Efficient processing of recommendation algorithms on a single-machine-based graph engine. J Supercomput 76, 7985–8002 (2020). https://doi.org/10.1007/s11227-018-2477-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-018-2477-4

Keywords

Search

Navigation