Sampling networks by the union of m shortest path trees

Abstract

Many network topology measurements capture or sample only a partial view of the actual network structure, which we call the underlying network. Sampling bias is a critical problem in the field of complex networks ranging from biological networks, social networks and artificial networks like the Internet. This bias phenomenon depends on both the sampling method of the measurements and the features of the underlying networks. In RIPE NCC and the PlanetLab measurement architectures, the Internet is mapped as $G_{{\cup }_m\mathit{spt}}$, the union of shortest paths between each pair of a set $\mathcal{M}$ of m testboxes, or equivalently, m shortest path trees. In this paper, we investigate this sampling method on a wide class of real-world complex networks as well as on the weighted Erdös–Rényi random graphs. This general framework examines the effect of the set of testboxes on $G_{{\cup }_m\mathit{spt}}$. We establish the correlation between the subgraph $G_{\mathcal{M}}$ of the underlying network, i.e. the set $\mathcal{M}$ and the direct links between nodes of set $\mathcal{M}$, and the sampled network $G_{{\cup }_m\mathit{spt}}$. Furthermore, we illustrate that in order to obtain an increasingly accurate view of a given network, a higher than linear detection/measuring effort (the relative size $m/N$ of set $\mathcal{M}$) is needed, where N is the size of the underlying network. Finally, when the relative size $m/N$ of set $\mathcal{M}$ is small, we characterize the kind of networks possessing small sampling bias, which provides insights on how to place the testboxes for good network topology measurement.

Introduction

Topologies of complex networks ranging from biological networks such as gene regulatory networks [1], metabolic networks [2], artificial networks like the Internet, the WWW to social networks, e.g. paper citations and collaboration networks [3], have been accumulated by active investigation in recent years. However, many surveyed networks to date are, in fact, subnets of the actual network, which we call the “underlying network”. For example, only a subset of the molecular entities in a cell have been sampled in protein interaction, gene regulation and metabolic networks. The topology of the Internet is inferred by aggregating paths, which reveals only a part of the whole Internet. Thus, these identified networks are sampled networks of the underlying networks according to different mapping or sampling methods.

In this work, we study the bias phenomenon of a sampling method that originated from the Internet. The topology of the Internet has typically been measured by the union of sampling traceroutes [4], which are approximately shortest paths. Mainly two sampling methods exist: (a) The topology is built from the union of traceroutes from a small set of sources to a larger set of destinations as in the CAIDA skitter project [5] . The sampled map can be modeled as the union of the spanning trees rooted at the sources. (b) The traceroute measurements are carried out between each pair of a set $\mathcal{M}$ of m testboxes or testbeds. The sampled network, denoted as $G_{{\cup }_m\mathit{spt}}\text{,}$ is the union of m shortest path trees SPTs, where each SPT is the union of shortest paths from the root $\in \mathcal{M}$ to the other $m-1$ testboxes $\in \mathcal{M}$. Equivalently, $G_{{\cup }_m\mathit{spt}}$ is the union of shortest paths between each node pair in the set $\mathcal{M}$ of m testboxes. The RIPE NCC [6] and the PlanetLab [7] measurement architectures are examples of this type. The methodology in (a) has been argued and even proved to introduce such intrinsic biases that statistical properties of the sampled topology may sharply differ from that of the underlying graph (see e.g. [8], [9], [10]). While most related works on Internet exploration have been devoted to the sampling method (a), we investigate the other sampling method (b). Although the number of destinations may be limited to the number m of measurement boxes, the spurious effects in (a), where nodes and links closer to the sources are more likely to be sampled than those surrounding the destinations, can be reduced.

With statistical and graph theory methodologies, we investigate this sampling method (m shortest path trees) on a wide class of networks: the weighted Erdös–Rényi random graphs, which represent dense and homogeneous networks, and the unweighted real-world complex networks which are generally sparse and inhomogeneous graphs. Various underlying networks are investigated, because network sampling is a generic problem residing in various disciplines and the actual underlying network topology is mostly uncertain. Here, we focus on the sampling bias (the incompleteness of the network mapping) introduced purely by the sampling method. Technical limitations in the topology measurements may also introduce significant sampling bias. For example, the network measured by traceroute represents the interconnections of IP addresses. The bias in mapping the router level Internet topology depends highly on the alias resolution technique, which maps IP addresses to the corresponding routers [11]. Such specific technical concerns, which vary in the measuring of different complex networks, are not explored in this paper.

The sampled network $G_{{\cup }_m\mathit{spt}}$ depends on the set $\mathcal{M}$ of m boxes as well as the underlying network. In this work, we focus on the effect of the testboxes, in particular, (1) the subgraph $G_{\mathcal{M}}$ of the underlying network, consisting of the set $\mathcal{M}$ and the direct links between nodes of set $\mathcal{M}$, and (2) the relative size $m/N$ of set $\mathcal{M}$, where N is the size of the underlying network. With a given set of testboxes, the sampling bias varies for different networks. The kind of networks with small sampling bias will also be briefly mentioned in this paper.

The main contributions of this study can be summarized as follows:

• Introduction of a general framework for network sampling on both weighted and unweighted complex networks.

• Establishment of the correlation between the interconnections of set $\mathcal{M}$, i.e. the subgraph $G_{\mathcal{M}}$, and the sampled network $G_{{\cup }_m\mathit{spt}}$.

• Illustration of the detection/measuring effort (the relative size $m/N$ of set $\mathcal{M}$) to obtain an increasingly accurate view of a given network.

• Characterization of networks bearing small sampling bias when $m/N$ is small and the corresponding proposal of testbox placement for good network topology measurements.