A resource allocation policy for delay minimization in fetching capacitated feeds

Abstract

As social media services such as Twitter and Facebook are gaining popularity, the amount of information published from those services is explosively growing. Most of them use feeds to facilitate distribution of a huge volume of content they publish. In this context, many users subscribe to feeds to acquire up-to-date information through feed aggregation services, and recent real-time search engines also increasingly utilize feeds to promptly find recent web content when it is produced. Accordingly, it is necessary for such services to effectively fetch feeds for minimizing fetching delay, while at the same time maximizing the number of fetched entries. Fetching delay is a time lag between entry publication and retrieval, which is primarily incurred by finiteness of fetching resources. In this paper, we consider a polling-based approach among the methods applicable to fetching feeds, which bases on a specific schedule for visiting feeds. While the existing polling-based approaches have focused on the allocation of fetching resources to feeds in order to either reduce the fetching delay or increase the number of fetched entries, we propose a resource allocation policy that can optimize both objectives. Extensive experiments have been carried out to evaluate the proposed model, in comparison with the existing alternative methods.