Extracting Positive and Negative Keywords for Web Communities

Abstract

The linkage information is shown to be useful to find goodWeb pages at a search engine [5],[10]. But, in general, a search result contains several topics. Clustering Web pages enables a user to browse them easily. There are several works on clustering Web pages [7,10-12]. In [9], we visualizedWeb graphs using spring model. But clustering is not enough to understand the topics of the clusters. Extraction of meta-data that explains communities is an important subject. Chakrabarti et al [6] used the terms in the small neighborhood around a document. Our approach is to combine the clustering and keyword extraction to interpret the communities.

To find communities, we solve the eigensystem of the matrix made from the link structure of Web pages. To get characteristic keywords from found communities, we use the algorithm developed in [1, 2, 8]. The input for the algorithm are two sets of documents - positive and negative documents. The algorithm outputs a pattern which well classifies them. This algorithm is robust for errors and noises, so that it is suitable for Web pages. The novelty of the keyword extraction algorithm is that keywords not only characterize one community but also distinguish the community from others. Thus, even if we fix a community, we have different characteristic keywords for the community according to the counter part.

We found good characteristic keywords from two communities without seeing Web pages in them. We also show an experimental result in which different keywords are extracted according to the counter part.