Network-aware summarisation for resource discovery in P2P-content networks

Abstract

Many application scenarios have a read-dominated behaviour of information provision, which means that there are few updates, and that users execute frequent queries for information discovery. Discovery of content in such systems could benefit from the use of summary techniques in order to facilitate the discovery process and minimise the size of the data exchanged. This is particularly applicable within Grid computing environments where nodes (representing a group of computational and storage resources), which enable jobs to reach them may need to exchange metadata with each other about their resources. We introduce network-aware summarisation algorithms for the resource discovery in P2P-content networks, which are based on Cobweb clustering. We identify how summarisation can improve the discovery process while also improving the accuracy of the discovered resource(s). Metrics based on precision–recall are used to compare the accuracy for specific types of queries generated over the summarised content.

Introduction

Large-scale information systems have gained in importance over recent years, often treating with many concurrent users and managing an increasing amount of information. Different systems address the scalability challenges that arise within such systems. First, using a Peer-to-Peer (P2P) systems provides scalability in terms of the number of participants accessing such systems. Second, summarisation techniques help reduce the amount of information exchanged between system nodes. Thus, a combination of these two techniques is a promising solution for Grid-based Information Systems [1], [2] and Distributed Market Information Systems [3].

Many scientific applications involve the execution of not just a single task but a coordinated execution of multiple tasks—with data dependences existing between them. Such tasks may either be independent components or complete applications that require a combination of jobs. A task graph is generated in which vertices represent tasks, and arcs represent data dependences between tasks. An emphasis on the use of scientific workflow engines in computational science over recent years (such as Taverna [4] and Kepler [5] in the BioSciences) has led to combining the output of tasks that are executed across different (often distributed) platforms. For instance, an application such as the TeraGrid or EGEE/EGI may be distributed across multiple nodes of a Grid infrastructure and requires data to be transferred between nodes across a network. Taking account of data transfer rates between tasks must therefore be an important criteria to meet an overall application makespan when considering where to place tasks across a distributed infrastructure. Data transfer rates are ignored in resource discovery systems, which often look at the properties of a single task rather than the workflow in its entirety. In Grid system registries, for instance, resource properties are relatively stable, in that they will provide the same operating system and hardware configuration over a long time frame. However, the network connectivity between such resources may vary considerably, depending on the jobs being executed and the associated data transfer between them. Although new resources may also appear in such systems, queries to discover suitable resources to execute a job are likely to result in a much greater number than the new resources being added/removed to/from the system [6]. Thus, the efficiency of the data retrieval is more important than the costs of the setup.

This paper proposes a technique based on data summarisation that can take account of data transfer rates between jobs. The main contribution of this paper is the application of Cobweb clustering to generate summaries of resource properties, thereby supporting scalability of P2P-based content networks. The scalability is achieved by providing efficient network-aware data discovery and reducing the time and amount of messages that are exchanged between P2P nodes. This paper analyses the behaviour of the proposed mechanism using simulations with up to half a million resources, each having several attributes that are based on PlanetLab evaluations.

Summarisation techniques have been used in several research areas, for instance, database management [7], video coding [8], [9], [10], [11], or visualisation of web pages [12]. These applications demonstrate the usefulness of reducing the amount of information (but not the quality of such information) when performing several tasks.

The remainder of this paper is organised in several sections. Section 2 presents the background of content networks, and Section 3 explains the Cobweb-based data summarisation technique. Afterwards, the system architecture, which utilises summaries to discover resources is introduced in Section 4. The system architecture includes a summary-based resource discovery technique with an extension of network awareness. The evaluation for a network-aware summarisation environment that is efficient and scalable is presented in Section 5. Section 6 compares the proposed network-aware summarisation for large-scale information systems with the existing related work. Finally, the conclusions and the future work are covered in Section 7.