Skip to main content
SpringerLink
Log in

Parallel Vector Computing Technique for Discovering Communities on the Very Large Scale Web Graph

  • Conference paper
Data Warehousing and Knowledge Discovery (DaWaK 2003)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2737))

Included in the following conference series:

Abstract

The study of the authoritative pages and community discovery from an enormous Web contents has attracted many researchers. One of the link-based analysis, the HITS algorithm, calculates authority scores as the eigenvector of a adjacency matrix created from the Web graph. Although it was considered impossible to compute the eigenvector of a very large scale of Web graph using previous techniques, due to this calculation requires enormous memory space. We make it possible using data compression and parallel computation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. AlltheWeb.com, http://www.alltheweb.com/

  2. Barrett, R., Chan, T., Donato, J., Berry, M., Demmel, J.: Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods, 2nd edn. SIAM, Philadelphia (1994)

    Google Scholar 

  3. Bharat, K., Henzinger, M.: Improved Algorithm for Topic Distillation in a Hyperlinked Environment. In: Proc. of ACM SIGIR 1998, Melbourne, Australia, pp. 104–111 (1998)

    Google Scholar 

  4. Broder, A., Kumar, R., Maghoul, F., Raghavan, P., Rajagopalan, S., Stata, R., Tomkins, A., Wiener, J.: Graph Structure in the Web. Computer Networks, 309–320 (2000)

    Google Scholar 

  5. Chakrabarti, S., Dom, B., Kumar, S., Raghavan, P., Rajagopalan, S., Tomkins, A., Gibson, D., Kleinberg, J.: Mining the Web’s Link Structure. Computer, 60–67 (1999)

    Google Scholar 

  6. Dean, J., Henzinger, M.: Finding Related Pages in the World Wide Web. In: Proc. of the 8th World-Wide Web Conference, Amsterdam, Netherlands (1999)

    Google Scholar 

  7. Fuller, L., Bechtel, R.: Introduction to matrix algebra. Dickenson Pub. Co. (1967)

    Google Scholar 

  8. Google, http://www.google.com/

  9. Hirokawa, S., Ikeda, D.: Structural Analysis of Web Graph. Transactions of the Japanses Society for Artificial Intelligence 16(4), 525–529 (2001)

    Google Scholar 

  10. Internet Software Consortium, http://www.isc.org/

  11. Kazama, K., Harada, M.: Advanced Web Search Engine Technologies. Transactions of the Japanses Society for Artificial Intelligence 16(4), 503–508 (2001)

    Google Scholar 

  12. Kleinberg, J.: Authoritative Sources in a Hyperlinked Environment. Journal of the ACM, 604–632 (1999)

    Google Scholar 

  13. Kleinberg, J., Kumar, R., Raghavan, P., Rajagopalan, S., Tomkins, A.: The Web as a Graph: Measurements, Models, and Methods. In: Asano, T., Imai, H., Lee, D.T., Nakano, S.-i., Tokuyama, T. (eds.) COCOON 1999. LNCS, vol. 1627, pp. 1–17. Springer, Heidelberg (1999)

    Chapter  Google Scholar 

  14. Kumar, R., Raghavan, P., Rajagopalan, S., Tomkins, A.: Extracting Large-scale Knowledge Bases from the Web. In: Proceedings of the 25th International Conference on Very Large Databases, Edinburgh, UK, pp. 639–650 (1999)

    Google Scholar 

  15. Murata, T.: Discovery of Web Communities Based on Co-occurrence of References. Transactions of the Japanses Society for Artificial Intelligence 16(3), 316–323 (2001)

    Article  Google Scholar 

  16. NII-NACSIS Test Collection for IR Systems Project, http://research.nii.ac.jp/ntcadm/indexen.html

  17. Openfind, http://www.openfind.com/

  18. Page, L., Brin, S., Motwani, R., Winograd, T.: The PageRank Citation Ranking: Bringing Order to the Web, Stanford Digital Library Project No. 1999-66 (1999), http://dbpubs.stanford.edu/pub/1999-66

  19. Yamada, S., Murata, T., Kitamura, Y.: Intelligent Web Information System. Transactions of the Japanses Society for Artificial Intelligence 16(4), 495–502 (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Systems Science, Graduate School of Informatics, Kyoto University,  

    Kikuko Kawase & Hiroyuki Kawano

  2. Academic Center for Computing and Media Studies, Kyoto University,  

    Minoru Kawahara, Takeshi Iwashita & Masanori Kawazawa

Authors
  1. Kikuko Kawase
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Minoru Kawahara
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Takeshi Iwashita
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hiroyuki Kawano
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Masanori Kawazawa
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Graduate School of Informatics, Kyoto University, Yoshida-Honmachi, Sakyo, 606-8501, Kyoto, Japan

    Yahiko Kambayashi

  2. I.B.M. India Research Lab, India

    Mukesh Mohania

  3. Institute for Application Oriented Knowledge Processing (FAW), Johannes Kepler University Linz, Austria

    Wolfram Wöß

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Kawase, K., Kawahara, M., Iwashita, T., Kawano, H., Kawazawa, M. (2003). Parallel Vector Computing Technique for Discovering Communities on the Very Large Scale Web Graph. In: Kambayashi, Y., Mohania, M., Wöß, W. (eds) Data Warehousing and Knowledge Discovery. DaWaK 2003. Lecture Notes in Computer Science, vol 2737. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45228-7_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-45228-7_16

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-40807-9

  • Online ISBN: 978-3-540-45228-7

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation