Abstract
Recently published studies have shown that Latent Semantic Indexing (LSI) plays an important role in content-based full text information retrieval of P2P system. However, it is a challenging problem to generate global consistent LSI structures in P2P systems because their nodes are self-organizing and their corpora are large, dynamic and distributed on different nodes. In this paper we propose a method for building LSI structures from distributed corpora. Our method is consisted with a network model for semantic information sampling and exchanging and a Reduced-Dimension-Representation (RDR)s merging algorithm. By the signal and noise subspace model, we also provide a theoretical justification that the RDR merging algorithm is sound. A simple numerical experiment shows that our RDR merging algorithm can keep query precision on an acceptable level.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Tang, C., Xu, Z., Dwarkadas, S.: Peer-to-Peer Information Retrieval Using Self-Organizing Semantic Overlay Networks. In: SIGCOMM 2003 (2003)
Tang, C., Xu, Z., Dwarkadas, S.: On Scaling Latent Semantic Indexing for Large Peer-to-Peer Systems. In: Proceedings of the 27th annual international conference on Research and development in information retrieval, pp. 112–121 (2004)
Shen, H.T., Shu, Y.F., Yu, B.: Efficient Semantic-Based Content Search in P2P Network. In: IEEE Transactions on Knowledge and Data Engineering archive, vol. 16, pp. 813–826 (2004)
Berry, M., Drmac, Z., Jessup, E.: Matrices, Vector Spaces, and Information Retrieval. SIAM Review 41(2), 335–362 (1999)
Berry, M.W., Dumais, S.T., O’Brien, G.W.: Using linear algebra for intelligent information retrieval. SIAM Review 37(4), 573–595 (1995)
Deerwester, S., Dumais, S.T.: Furnas. Indexing by latent semantic analysis. Journal of the American Society for Information Science 41(6), 391–407 (1990)
Zha, H., Simon, H.: On Updating Problems in Latent Semantic Indexing. SIAM Journal of Scientific Computing 21, 782–791 (1999)
Ratnasamy, S., Francis, P., Handley, M., Karp, R., Shenker, S.: A Scalable Content-Addressable Network. In: Proc. SIGCOMM (2001)
Stoica, I., Morris, R., Karger, D., Kaashoek, F., Balakrishnan, H.: Chord: A Scalable Peer-to-Peer Lookup Service for Internet Applications. In: Proc. SIGCOMM (2001)
Rowstron, A., Druschel, P.: Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer systems. IFIP/ACM Middleware, Heidellberg, Germany, (November 2001)
Yang, B., Garcia-Molina, H.: Designing a super-peer network. In: ICDE (2003)
Cornell Smart System, ftp://ftp.cs.cornell.edu/pub/smart
TMG, http://scgroup.hpclab.ceid.upatras.gr/scgroup/Projects/TMG/
SVDPACKC, http://www.netlib.org/svdpack/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhang, S., Wu, G., Chen, G., Xu, L. (2005). On Building and Updating Distributed LSI for P2P Systems. In: Chen, G., Pan, Y., Guo, M., Lu, J. (eds) Parallel and Distributed Processing and Applications - ISPA 2005 Workshops. ISPA 2005. Lecture Notes in Computer Science, vol 3759. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11576259_2
Download citation
DOI: https://doi.org/10.1007/11576259_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29770-3
Online ISBN: 978-3-540-32115-6
eBook Packages: Computer ScienceComputer Science (R0)