Abstract
Collaborative filtering (CF) has proved to be one of the most effective information filtering techniques. However, as their calculation complexity increased quickly both in time and space when the record in user database increases, traditional centralized CF algorithms has suffered from their shortage in scalability. In this paper, we first propose a novel distributed CF algorithm called PipeCF through which we can do both the user database management and prediction task in a decentralized way. We then propose two novel approaches: significance refinement (SR) and unanimous amplification (UA), to further improve the scalability and prediction accuracy of PipeCF. Finally we give the algorithm framework and system architecture of the implementation of PipeCF on Peer-to-Peer (P2P) overlay network through distributed hash table (DHT) method, which is one of the most popular and effective routing algorithm in P2P. The experimental data show that our distributed CF algorithm has much better scalability than traditional centralized ones with comparable prediction efficiency and accuracy.
Supported by the National Natural Science Foundation of China under Grant No. 60372078
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Goldberg, D., Nichols, D., Oki, B.M., Terry, D.: Using collaborative filtering to weave an information tapestry. Communications of the ACM 35(12), 61–70 (1992)
Herlocker, J.L., Konstan, J.A., Borchers, A., Riedl, J.: An algorithmic framework for performing collaborative filtering. In: Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, pp. 230–237 (1999)
Breese, J., Heckerman, D., Kadie, C.: Empirical Analysis of Predictive Algorithms for Collaborative Filtering. In: Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence, pp. 43–52 (1998)
Resnick, P., Iacovou, N., Suchak, M., Bergstrom, P., Riedl, J.: GroupLens: an open architecture for collaborative filtering of netnews. In: Proceedings of the 1994 ACM conference on Computer supported cooperative work, October 22-26, pp. 175–186. Chapel Hill, North Carolina (1994)
Shardanand, U., Maes, P.: Social information filtering: algorithms for automating “word of mouth”. In: Proceedings of the SIGCHI conference on Human factors in computing systems, Denver, Colorado, United State, May 7-11, pp. 210–217 (1995)
Eachmovie collaborative filtering data set, http://research.compaq.com/SRC/eachmovie
Tveit, A.: Peer-to-peer based Recommendations for Mobile Commerce. In: Proceedings of the First International Mobile Commerce Workshop, July 2001, pp. 26–29. ACM Press, Rome (2001)
Olsson, T.: Bootstrapping and Decentralizing Recommender Systems, Licentiate Thesis 2003-006, Department of Information Technology, Uppsala University and SICS (2003)
Canny, J.: Collaborative filtering with privacy. In: Proceedings of the IEEE Symposium on Research in Security and Privacy, Oakland, CA, May 2002, pp. 45–57 (2002); IEEE Computer Society, Technical Committee on Security and Privacy, IEEE Computer Society Press
Ratnasamy, S., Francis, P., Handley, M., Karp, R., Shenker, S.: A scalable contentaddressable network. In: SIGCOMM (August 2001)
Stocal, I., et al.: Chord: A scalable peer-to-peer lookup service for Internet applications. In: ACM SIGCOMM, San Diego, CA, USA, pp. 149–160 (2001)
Rowstron, A., Druschel, P.: Pastry: Scalable, distributed object location and routing for large scale peer-to-peer systems. In: IFIP/ACM Middleware, Hedelberg, Germany (2001)
Zhao, B.Y., et al.: Tapestry: An infrastructure for fault-tolerant wide-area location and routing. Tech. Rep. UCB/CSB-0-114, UC Berkeley, EECS 2001 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Han, P., Xie, B., Yang, F., Wang, J., Shen, R. (2004). A Novel Distributed Collaborative Filtering Algorithm and Its Implementation on P2P Overlay Network. In: Dai, H., Srikant, R., Zhang, C. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2004. Lecture Notes in Computer Science(), vol 3056. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24775-3_13
Download citation
DOI: https://doi.org/10.1007/978-3-540-24775-3_13
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22064-0
Online ISBN: 978-3-540-24775-3
eBook Packages: Springer Book Archive