SUPNET: An end-to-end solution to scalable unstructured P2P networking

Abstract

Despite many improvements on original unstructured P2P networks, these systems still suffer from several problems, the most important of which are, (a) lack of guarantees on the integrity of the network topology in the face of churns, (b) excessive traffic cost and (c) poor quality of search results. This paper introduces an end-to-end scalable unstructured P2P networking solution called SUPNET to address many of these issues. SUPNET is based on our pragmatic, design oriented approach to engineering complex networks. Rather than modeling dynamical behavior in already-existing networks, we actively design and implement local stochastic dynamics so that an engineered global system, with predictable structures emerges. The resulting protocol, SUPNET, consists of two sub-protocols for network management and content search. The network management sub-protocol is scalable and highly robust and is capable of utilizing the heterogeneous distribution of network resources. Its high stability is the result of implementation of a novel distributed feedback mechanism. The search sub-protocol is capable of locating every item, even if a single copy of that item exists in the network, while producing a traffic that scales provably sub-linear with the network size. It also contains mechanisms for very efficient location of popular items as well as distributed parameter tuning algorithms. These, along with inherently self-organized and de-centralized operation, relative ease of implementation and solid analytical foundation, make SUPNET a compelling solution for unstructured P2P networking.

Appendix

Theoretical preliminaries

• (1) Mixing Times of Random Networks: Random walk on any connected graph eventually mixes, i.e., the probability that the random walker is found at a node with degree k will converge to ck for some constat c independent of k, as the length of the walk goes to infinity. This, of course, is the well known linear preferential attachment. If the random walk of TSA mixes, it will return a node with degree k is linearly proportional to k for large L.

  The length of the walk required for mixing can be as high as O(N ³), where N is the number of nodes (see e.g., [3] for a review of classic results). Recently, it has been shown that for many randomly generated networks, the value of L for the random walk to mix is only O(logN) [17]. Therefore, initiating connections by sampling nodes through a short random walk of length L = O(logN) closely approximates linear preferential attachment. Random-walk as a means for approximating preferential attachment has been employed in other works before (see e.g. [9]). We have resorted to extensive simulations to verify that the mixing time for SUPNET-T networks is indeed O(logN), even though there is no mathematical proof at this time.

• (2) Percolation Threshold of Random Power-Law Networks: Percolation threshold of a network is defined as the critical threshold for which percolation with a smaller probability results in a network with no giant connected component. The percolation probability of random networks on a given degree distribution can be shown to be [24, 25]:

  $$\label{eqn:pc} p_c=\frac{\langle k \rangle}{\langle k^2 - \langle k \rangle\rangle} $$

  (6)

  where ${\langle k \rangle}$ and ${\langle k^2 \rangle}$ are the average and variance of the degree distribution. For a random power-law network with exponent τ, this evaluates to:

  $$ p_c= \frac{\sum_{k=1}^{k_{max}} k\cdot k^{-\tau}}{\sum_{k=1}^{k_{max}} k^2\cdot k^{-\tau}-\sum_{k=1}^{k_{max}} k\cdot k^{-\tau}} $$

  where k _max is the maximum degree. For large k _max and 2 < τ < 3, the average degree is a constant, while the variance diverges. As such, p _c can be approximated as, $p_c\approx A k_{max}^{\tau-3}$ for some constant A. The cut-off value of k _max is ~N^1/τ (at which point the expected number of nodes with degree k _max becomes in the order of 1). Plugging this into p _c we get, $p_c \sim N^{-\frac{3-\tau}{\tau}}$ for 2 < τ < 3.

• (3) Scaling Properties of PSA: The following theorem about the search traffic of PSA has been proved in [4]:

  Theorem 1 For τ = 2, PSA with high probability (w.h.p.) can find any content (even if there is a single copy of the content in the network) within latency of only O(logN) requiring an average cache size of only O(logN). For τ = 2, the percolation probability should be chosen as κlog(N)N^− 1/2, for some proper constant κ in the order of one and a walk length of L = O(logN). Thus, the number of messages per query is only O(logNN^1/2). For 2 < τ < 3, on the other hand, the search time is still O(logN), while the average cache size is O(N^{1 − 2/τ)}, producing a traffic per query of only O(N^{2 − 3/τ}). The percolation probability should be chosen as κ′N^{1 − 3/τ}, for some proper κ′ > 0 in the order of one and a walk length of L = O(N^{1 − 2/τ}).