Article

Focused crawling by exploiting anchor text using decision tree

Authors:

Jun Li,

Kazutaka Furuse,

Kazunori YamaguchiAuthors Info & Claims

WWW '05: Special interest tracks and posters of the 14th international conference on World Wide Web

Pages 1190 - 1191

https://doi.org/10.1145/1062745.1062933

Published: 10 May 2005 Publication History

Get Access

Abstract

Focused crawlers are considered as a promising way to tackle the scalability problem of topic-oriented or personalized search engines. To design a focused crawler, the choice of strategy for prioritizing unvisited URLs is crucial. In this paper, we propose a method using a decision tree on anchor texts of hyperlinks. We conducted experiments on the real data sets of four Japanese universities and verified our approach.

References

[1]

Soumen Chakrabarti, Martin van den Berg, and Byron Dom. Focused crawling: a new approach to topic-specific Web resource discovery. In The Eighth International World Wide Web Conference, Toronto, Canada, May 1999.

Digital Library

Google Scholar

[2]

M. Diligenti, F. M. Coetzee, S. Lawrence, C. L. Giles, and M. Gori. Focused Crawling Using Context Graphs. In Proc. of the 26th VLDB Conf., pp. 527--534, 2000.

Digital Library

Google Scholar

[3]

J. R. Quinlan. Induction of decision trees. Machine Learning, Vol. 1, No. 1, pp. 81--106, 1986.

Digital Library

Google Scholar

Cited By

View all

Joe Dhanith PSaeed KRohith GRaja S(2024)Weakly supervised learning for an effective focused web crawlerEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.107944132:COnline publication date: 18-Jul-2024
https://dl.acm.org/doi/10.1016/j.engappai.2024.107944
Rajiv SNavaneethan C(2022) A supervised learning‐based approach for focused web crawling for IoMT using global co‐occurrence matrix Expert Systems10.1111/exsy.1299340:4Online publication date: 28-Mar-2022
https://doi.org/10.1111/exsy.12993
Prabha KMahesh CGoundar SRaja S(2022)Amelioration of linguistic semantic classifier with sentiment classifier manacle for the focused web crawlerInternational Journal of Information Technology10.1007/s41870-022-01139-w15:2(1137-1149)Online publication date: 27-Dec-2022
https://doi.org/10.1007/s41870-022-01139-w
Show More Cited By

Index Terms

Focused crawling by exploiting anchor text using decision tree
1. Computing methodologies
  1. Artificial intelligence
    1. Search methodologies
      1. Discrete space search
      2. Game tree search
2. Information systems
  1. Information retrieval
    1. Information retrieval query processing

Recommendations

Current challenges in web crawling
ICWE'13: Proceedings of the 13th international conference on Web Engineering

Web crawling, a process of collecting web pages in an automated manner, is the primary and ubiquitous operation used by a large number of web systems and agents starting from a simple program for website backup to a major web search engine. Due to an ...
Focused crawling guided by link context
AIA'06: Proceedings of the 24th IASTED international conference on Artificial intelligence and applications

In designing a focused crawler, the choice of strategy for prioritizing unvisited URLs is crucial. There is an intuition that the text surrounding a link or the link context on the HMTL page is a good summary of the target page. But little work has been ...
Exploiting Multiple Features with MEMMs for Focused Web Crawling
NLDB '08: Proceedings of the 13th international conference on Natural Language and Information Systems: Applications of Natural Language to Information Systems

Focused web crawling traverses the Web to collect documents on a specific topic. This is not an easy task, since focused crawlers need to identify the next most promising link to follow based on the topic and the content and links of previously crawled ...

Comments

Information & Contributors

Information

Published In

WWW '05: Special interest tracks and posters of the 14th international conference on World Wide Web

May 2005

454 pages

ISBN:1595930515

DOI:10.1145/1062745

Conference Chairs:
Allan Ellis
Southern Cross University, Australia
,
Tatsuya Hagino
Keio University, Japan
,
Program Chairs:
Fred Douglis
IBM Research
,
Prabhakar Raghavan
Verity, Inc.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 May 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
524
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)1

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Joe Dhanith PSaeed KRohith GRaja S(2024)Weakly supervised learning for an effective focused web crawlerEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.107944132:COnline publication date: 18-Jul-2024
https://dl.acm.org/doi/10.1016/j.engappai.2024.107944
Rajiv SNavaneethan C(2022) A supervised learning‐based approach for focused web crawling for IoMT using global co‐occurrence matrix Expert Systems10.1111/exsy.1299340:4Online publication date: 28-Mar-2022
https://doi.org/10.1111/exsy.12993
Prabha KMahesh CGoundar SRaja S(2022)Amelioration of linguistic semantic classifier with sentiment classifier manacle for the focused web crawlerInternational Journal of Information Technology10.1007/s41870-022-01139-w15:2(1137-1149)Online publication date: 27-Dec-2022
https://doi.org/10.1007/s41870-022-01139-w
Shrivastava GPateriya RKaushik P(2022)An efficient focused crawler using LSTM-CNN based deep learningInternational Journal of System Assurance Engineering and Management10.1007/s13198-022-01808-w14:1(391-407)Online publication date: 19-Dec-2022
https://doi.org/10.1007/s13198-022-01808-w
Songtao SHuaiguang WJiangtao M(2019)An Improved Focused Web Crawler based on Hybrid SimilarityInternational Journal of Performability Engineering10.23940/ijpe.19.10.p10.2645265615:10(2645)Online publication date: 2019
https://doi.org/10.23940/ijpe.19.10.p10.26452656
Wang JDeng SWang L(2019)Multilingual Focused Crawler System based on Web Content Extraction and Path ConfigurationIOP Conference Series: Materials Science and Engineering10.1088/1757-899X/569/5/052030569(052030)Online publication date: 9-Aug-2019
https://doi.org/10.1088/1757-899X/569/5/052030
Hassan TCruz CBertaux A(2017)Ontology-based approach for unsupervised and adaptive focused crawlingProceedings of The International Workshop on Semantic Big Data10.1145/3066911.3066912(1-6)Online publication date: 19-May-2017
https://dl.acm.org/doi/10.1145/3066911.3066912
Hassan TCruz CBertaux A(2017)Predictive and evolutive cross-referencing for web textual sources2017 Computing Conference10.1109/SAI.2017.8252230(1114-1122)Online publication date: Jul-2017
https://doi.org/10.1109/SAI.2017.8252230
Suebchua TManaskasemsak BRungsawang AYamana H(2017)Efficient Topical Focused Crawling Through Neighborhood FeatureNew Generation Computing10.1007/s00354-017-0029-836:2(95-118)Online publication date: 15-Dec-2017
https://doi.org/10.1007/s00354-017-0029-8
Kumar MBhatia RRattan D(2017)A survey of Web crawlers for information retrievalWIREs Data Mining and Knowledge Discovery10.1002/widm.12187:6Online publication date: 7-Aug-2017
https://doi.org/10.1002/widm.1218
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Current challenges in web crawling

Focused crawling guided by link context

Exploiting Multiple Features with MEMMs for Focused Web Crawling

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations