Abstract
Collaborative filtering is one of the most popular recommendation techniques. While the quality of the recommendations has been significantly improved in the last years, most approaches present poor efficiency and scalability. In this paper, we study several factors that affect the performance of a k-Nearest Neighbors algorithm, and we propose a distributed architecture that significantly improves both throughput and response time. Two techniques for distributing recommender systems, user and item partition, were proposed and evaluated using that simulation model. We have found that user partition is generally better, with a faster response time and higher throughput.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Badue, C.S., Baeza-Yates, R., Ribeiro-Neto, B., Ziviani, A., Ziviani, N.: Analyzing imbalance among homogeneous index servers in a web search system. Inf. Process. Manage. 43, 592–608 (2007). http://dl.acm.org/citation.cfm?id=1224561.1224707
Bennett, J., Lanning, S.: The netflix prize.In:proceedings of KDD cup and workshop, KDDCup ’07, pp 3–6 (2007). ACM, San Jose,California, USA
Breese, J.S., Heckerman, D., Kadie, C.: Empirical analysis of predictive algorithms for collaborative filtering.In:proceedings of the fourteenth conference on uncertainty in artificial intelligence, UAI’98, pp 43–52. Morgan Kaufmann Publishers Inc., San Francisco (1998). URL http://dl.acm.org/citation.cfm?id=2074094.2074100
Büttcher, S., Clarke, C., Cormack, G.V.: Information retrieval: implementing and evaluating search engines. The MIT Press (2010)
Cacheda, F., Carneiro, V., Fernández, D., Formoso, V.: Comparison of collaborative filtering algorithms: Limitations of current techniques and proposals for scalable, high-performance recommender systems. ACM Trans. Web 5, 2:1–2:33 (2011). doi:10.1145/1921591.1921593
Cacheda, F., Carneiro, V., Plachouras, V., Ounis, I.: Performance analysis of distributed information retrieval architectures using an improved network simulation model. Inf. Process. Manage. 43(1), 204–224 (2007). doi:10.1016/j.ipm.2006.06.002
Cöster, R., Svensson, M.: Inverted file search algorithms for collaborative filtering. In Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR ’02, pp 246–252 (2002). ACM, New York, NY, USA
Desrosiers, C., Karypis, G. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) : Handbook, Recommender Systems, pp 107–144, Springer (2011)
Ding, S., Suel, T.: Faster top-k document retrieval using block-max indexes. In Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval, SIGIR ’11, pp 993–1002 (2011). ACM, New York, NY, USA
Formoso, V., Fernández, D., Cacheda, F., Carneiro, V.: Using neighborhood pre-computation to increase recommendation efficiency. In Proceedings of the fourth international conference on knowledge discovery and information retrieval (KDIR 2012). (To Appear) (2012)
Herlocker, J., Konstan, J.A., Riedl, J.: An empirical analysis of design choices in neighborhood-based collaborative filtering algorithms. Inf. Retr. 5(4), 287–310 (2002). doi:10.1023/A:1020443909834
Jiang, J., Lu, J., Zhang, G., Long, G.: Scaling-up item-based collaborative filtering recommendation algorithm based on hadoop. In Proceedings of the 2011 IEEE world congress on services, SERVICES ’11, pp 490–497. IEEE Computer Society, Washington (2011)
Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009). doi:10.1109/MC.2009.263
Linden, G., Smith, B., York, J.: Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Internet Computing 7(1), 76–80 (2003). doi:10.1109/MIC.2003.1167344
Little, M.C.: Javasim user’s guide. public release 0.3, version 1.0. (1999). http://javasim.codehaus.org
Manning, C.D., Raghavan, P., Schtze, H.: Introduction to Information Retrieval. Cambridge University Press, New York (2008)
Papagelis, M., Rousidis, I., Plexousakis, D., Theoharopoulos, E.: Incremental collaborative filtering for highly-scalable recommendation algorithms. In Proceedings of the 15th international conference on Foundations of Intelligent Systems, ISMIS’05, pp 553–561. Springer-Verlag, Berlin, Heidelberg (2005)
Resnick, P., Iacovou, N., Suchak, M., Bergstrom, P., Riedl, J.: Grouplens: an open architecture for collaborative filtering of netnews. In Proceedings of the 1994 ACM conference on Computer supported cooperative work, CSCW ’94, pp 175–186 (1994). ACM, New York, NY, USA
Ribeiro-Neto, B.A., Barbosa, R.A.: Query performance for tightly coupled distributed digital libraries. In Proceedings of the third ACM conference on digital libraries, DL ’98, pp 182–190 (1998). ACM, New York, NY, USA
Schelter, S., Boden, C., Markl, V.: Scalable similarity-based neighborhood methods with mapreduce. In Proceedings of the sixth ACM conference on Recommender systems, RecSys ’12, pp 163–170 (2012). ACM, New York, NY, USA
Turtle, H., Flood, J.: Inf. Process. Manage. 31(6), 831–850 (1995). doi:10.1016/0306-4573(95)00020-H
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Formoso, V., Fernández, D., Cacheda, F. et al. Distributed architecture for k-nearest neighbors recommender systems. World Wide Web 18, 997–1017 (2015). https://doi.org/10.1007/s11280-014-0298-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11280-014-0298-9