Abstract:
Extracting and processing information from Web pages is an important task in many areas like constructing search engines, information retrieval, and data mining from the ...Show MoreMetadata
Abstract:
Extracting and processing information from Web pages is an important task in many areas like constructing search engines, information retrieval, and data mining from the Web. A common approach in the extraction process is to represent a page as a "bag of words" and then to perform additional processing on such a flat representation. We propose a new, hierarchical representation that includes browser screen coordinates for every HTML object in a page. Using visual information one is able to define heuristics for the recognition of common page areas such as header, left and right menu, footer and center of a page. We show in initial experiments that using our heuristics defined objects are recognized properly in 73% of cases. Finally, we show that a Naive Bayes classifier, taking into account the proposed representation, clearly outperforms the same classifier using only information about the content of documents.
Date of Conference: 09-12 December 2002
Date Added to IEEE Xplore: 10 March 2003
Print ISBN:0-7695-1754-4