Abstract
Nowadays, user actions are tracked and recorded by multiple websites and e-commerce platforms, allowing them to better understand their preferences and support them with specific and accurate content suggestions. Researches have proposed several recommendation approaches and addressed several challenges such as data sparsity and cold start. However, the low-scalability problem remains a major challenge when handling large volumes of user actions data. This issue becomes more challenging when it comes to real-time applications. Such constraint requires a new class of low latency recommendation approaches capable of incrementally and continuously update their knowledge and models at scale as soon as data arrives. In this paper, we focus on the user-centered collaborative filtering as one of the most adopted recommendation approaches known for its lack of scalability. We propose two distributed and scalable implementations of collaborative filtering addressing the challenges and the requirements of batch offline and incremental online recommendation scenarios. Several experiments were conducted on a distributed environment using the MovieLens dataset in order to highlight the properties and the advantages of each variant.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Castagnos, S., Boyer, A.: Personalized communities in a distributed recommender system. In: Amati, G., Carpineto, C., Romano, G. (eds.) ECIR 2007. LNCS, vol. 4425, pp. 343–355. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-71496-5_32. http://dl.acm.org/citation.cfm?id=1763653.1763695
Chandramouli, B., Levandoski, J.J., Eldawy, A., Mokbel, M.F.: StreamRec: a real-time recommender system. In: Proceedings of the 2011 ACM SIGMOD International Conference on Management of Data, SIGMOD 2011, pp. 1243–1246. ACM, New York (2011). http://doi.acm.org/10.1145/1989323.1989465
Chang, S., et al.: Streaming recommender systems. In: Proceedings of the 26th International Conference on World Wide Web, WWW 2017, International World Wide Web Conferences Steering Committee, pp. 381–389. Republic and Canton of Geneva, Switzerland (2017). https://doi.org/10.1145/3038912.3052627
Chen, C., Yin, H., Yao, J., Cui, B.: TeRec: a temporal recommender system over tweet stream. Proc. VLDB Endow. 6(12), 1254–1257 (2013). https://doi.org/10.14778/2536274.2536289
Das, A.S., Datar, M., Garg, A., Rajaram, S.: Google news personalization: scalable online collaborative filtering. In: Proceedings of the 16th International Conference on World Wide Web, WWW 2007, pp. 271–280. ACM, New York (2007). http://doi.acm.org/10.1145/1242572.1242610
Diaz-Aviles, E., Drumond, L., Schmidt-Thieme, L., Nejdl, W.: Real-time top-N recommendation in social streams. In: Proceedings of the Sixth ACM Conference on Recommender Systems, RecSys 2012, pp. 59–66. ACM, New York (2012). http://doi.acm.org/10.1145/2365952.2365968
Domann, J., Lommatzsch, A.: A highly available real-time news recommender based on apache spark. In: Jones, J.F. (ed.) CLEF 2017. LNCS, vol. 10456, pp. 161–172. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-65813-1_17
Han, P., Xie, B., Yang, F., Shen, R.: A scalable P2P recommender system based on distributed collaborative filtering. Expert. Syst. Appl. 27(2), 203–210 (2004). https://doi.org/10.1016/j.eswa.2004.01.003
Han, P., Xie, B., Yang, F., Wang, J., Shen, R.: A novel distributed collaborative filtering algorithm and its implementation on P2P overlay network. In: Dai, H., Srikant, R., Zhang, C. (eds.) PAKDD 2004. LNCS (LNAI), vol. 3056, pp. 106–115. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24775-3_13
Harper, F.M., Konstan, J.A.: The movielens datasets: History and context. ACM Trans. Interact. Intell. Syst. 5(4), 19:1–19:19 (2015). http://doi.acm.org/10.1145/2827872
Huang, Y., Cui, B., Zhang, W., Jiang, J., Xu, Y.: TencentRec: real-time stream recommendation in practice. In: Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, SIGMOD 2015, pp. 227–238. ACM, New York (2015). http://doi.acm.org/10.1145/2723372.2742785
Kluver, D., Ekstrand, M.D., Konstan, J.A.: Rating-based collaborative filtering: algorithms and evaluation. In: Brusilovsky, P., He, D. (eds.) Social Information Access. LNCS, vol. 10100, pp. 344–390. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-90092-6_10
Koren, Y.: Factor in the neighbors: Scalable and accurate collaborative filtering. ACM Trans. Knowl. Discov. Data 4(1), 1:1–1:24 (2010). http://doi.acm.org/10.1145/1644873.1644874
Papagelis, M., Rousidis, I., Plexousakis, D., Theoharopoulos, E.: Incremental collaborative filtering for highly-scalable recommendation algorithms. In: Hacid, M.-S., Murray, N.V., Raś, Z.W., Tsumoto, S. (eds.) ISMIS 2005. LNCS (LNAI), vol. 3488, pp. 553–561. Springer, Heidelberg (2005). https://doi.org/10.1007/11425274_57
Sarwar, B., Karypis, G., Konstan, J., Riedl, J.: Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th International Conference on World Wide Web, WWW 2001, pp. 285–295. ACM, New York (2001). http://doi.acm.org/10.1145/371920.372071
Tveit, A.: Peer-to-peer based recommendations for mobile commerce. In: Proceedings of the 1st International Workshop on Mobile Commerce, WMC 2001, pp. 26–29. ACM, New York (2001). http://doi.acm.org/10.1145/381461.381466
Xie, B., Han, P., Yang, F., Shen, R.M., Zeng, H.J., Chen, Z.: DCFLA: a distributed collaborative-filtering neighbor-locating algorithm. Inf. Sci. 177(6), 1349–1363 (2007). https://doi.org/10.1016/j.ins.2006.09.005
Zanitti, M., Kosta, S., Sørensen, J.: A user-centric diversity by design recommender system for the movie application domain. In: Companion Proceedings of the The Web Conference 2018, pp. 1381–1389, WWW 2018. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland (2018). https://doi.org/10.1145/3184558.3191580
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Zaouali, K., Haddad, M.R., Zghal, H.B. (2018). Distributed Collaborative Filtering for Batch and Stream Processing-Based Recommendations. In: Panetto, H., Debruyne, C., Proper, H., Ardagna, C., Roman, D., Meersman, R. (eds) On the Move to Meaningful Internet Systems. OTM 2018 Conferences. OTM 2018. Lecture Notes in Computer Science(), vol 11229. Springer, Cham. https://doi.org/10.1007/978-3-030-02610-3_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-02610-3_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-02609-7
Online ISBN: 978-3-030-02610-3
eBook Packages: Computer ScienceComputer Science (R0)