Effective Filtering for Collaborative Publishing

Abstract

In little over the last decade the World Wide Web has established itself as a medium of interaction, communication, content delivery, and collaboration, opening doors of opportunity never before available to humanity, and on a scale unprecedented in human history. At the same time, information overload, due to democratization of content creation and delivery, remains a major problem. In this paper, we postulate that the problems of democracy are solved by democracy itself: harnessing the people power of the world wide web through collaborative filtering of content is the natural solution to the information overload problem; and we present approaches to promote such collaboration.

We show that the standard PageRank Algorithm, inspired by the effectiveness of citation-structure analysis (“all links are good, and the more the better”) to estimate the relative importance of articles in scientific literature, is becoming less effective in this increasingly democratized world of online content. As long as uniformly edited content produced by media companies and other corporate entities dominated online content, the topological similarity of the web to the world of scientific literature was maintained sufficiently well. The explosion of unedited blogs, discussion fora, and wikis, with their “messier” hyperlink structure, is rapidly reducing this similarity, and also the effectiveness of standard PageRank-based filtering methods.

We assume a slightly modified Web infrastructure in which links have positive and negative weights, and show that this enables radically different and more effective approaches to page ranking and collaborative content filtering, leading to a vastly improved environment to incentivize content creation and co-operation on the World Wide Web, helping realize, in essence, a vastly more efficient information economy in today’s online global village.