Optimizing cluster formation in super-peer networks via local incentive design

Abstract

A super-peer based overlay network architecture for peer-to-peer (P2P) systems allows for some nodes, known as the super-peers, that are more resource-endowed than others, to assume a higher share of workload. Ordinary peers are connected to the super-peers and rely on them for their transactional needs. Many criteria for a peer to choose its super-peer have been explored, some of them based on physical proximity, semantic proximity, or by purely random choice. In this paper, we propose an incentive-based criterion that uses semantic similarities between the content interests of the peers and, at the same time, encourages even load distribution across the super-peers. The incentive is achieved via a game theoretic framework that considers each peer as a rational player, allowing stable Nash equilibria to exist and hence guarantees a fixed point in the strategy space of the peers. This guarantees convergence (assuming static network parameters) to a locally optimal assignment of peers to super-peers with respect to a global cost that approximates the average query resolution time. We also show empirically that the local cost framework that we employ performs closely to (and in some cases better than) a similar scheme based on the formulation of a centralized cost function that requires the peers to know an additional global parameter.

Appendix

Proof of Theorem 1

We can express $C_{0}(\mathbf {r})$ as the aggregation of four partial sums: i) cost of peer l itself; ii)sum of costs of peers that belong to super-peer $k_{1}$ except l; iii) sum of costs of peers that belong to super-peer $k_{2}$ except l; and iv) sum of costs of those peers that belong neither to super-peer $k_{1}$ nor to super-peer $k_{2}$. In the three steps given below, we show that i), ii) and iii) decrease as peer l changes its assignment to minimize its cost (peer-level) while iv) is left unchanged. Thus, $C_{0}(\textbf {r})$ decreases, contradicting our assumption that $\hat {\mathbf {r}}$ is the global optimum (minimum) solution of $C_{0}$. We can divide the objective function as follows:

$$ \begin{array}{rll} C_{0}(\textbf{r})&= &\left(\frac{b_{l} }{w_{r_{l} }}\sum\limits_{\substack{j:r_{j} =r_{l} \\j\neq l}}b_{j} + \frac{\mu}{2}\sum\limits_{j:r_{j} \neq r_{l} } c_{lj}\right) \\&& +\sum\limits_{\substack{i:i\neq l\\r_{i} =k_{1} }} \left(\frac{b_{i} }{w_{r_{i} }}\sum\limits_{\substack{j:r_{j} =r_{i} \\j\neq i}}b_{j} + \frac{\mu}{2}\sum\limits_{j:r_{j} \neq r_{i} } c_{ij}\right) \\ &&+\sum\limits_{\substack{i:i\neq l\\r_{i} =k_{2} }} \left(\frac{b_{i} }{w_{r_{i} }}\sum\limits_{\substack{j:r_{j} =r_{i} \\j\neq i}}b_{j} + \frac{\mu}{2}\sum\limits_{j:r_{j} \neq r_{i} } c_{ij} \right) \\ &&+\sum\limits_{\substack{i:i\neq l,r_{i} \neq k_{1} \\r_{i} \neq k_{2} }} \left(\frac{b_{i} }{w_{r_{i} }}\sum\limits_{\substack{j:r_{j} =r_{i} \\j\neq i}}b_{j} + \frac{\mu}{2}\sum\limits_{j:r_{j} \neq r_{i} } c_{ij}\right)\end{array} $$

Thus,

$$ \begin{array}{rll} C_{0}(\textbf{r}) &= &C_{l} (\textbf{r}) + \sum\limits_{\substack{i:i\neq l\\r_{i} =k_{1} }} C_{i} (\textbf{r}) \qquad+ \sum\limits_{\substack{i:i\neq l\\r_{i} =k_{2} }} C_{i} (\textbf{r}) \\ &&+ \sum\limits_{\substack{i:i\neq l,r_{i} \neq k_{1} \\r_{i} \neq k_{2} }} C_{i} (\textbf{r}).\end{array} $$

The first term is the cost of peer l, the second term is sum of the costs of the peers that are assigned to the former super-peer of peer l (except peer l), the third term is the sum of the costs of the peers that belong to the prospective super-peer of peer l (except peer l) and the fourth term is the sum of the costs of the peers that belong neither to the current super-peer $k_{1}$ of peer l nor to the prospective super-peer $k_{2}$ of peer l.

Let the new assignment vector be $\textbf {r}^*=(r_{1} ^*,r_{2} ^*,...,r_{N} ^*)$, where $r_{i} ^*=\hat {r}_{i}$ $\forall ~i \neq l$.

$$\begin{array}{rll} C_{0}(\mathbf{\textbf{r}}^*) - C_{0}(\hat{\mathbf{r}}) &=& \Bigl( C_{l}(\mathbf{\textbf{r}}^*)- \notag C_{l}(\hat{\mathbf{r}})\Bigr) \\&&+ \left(\sum\limits_{\substack{i:i\neq l\\r^*_{i} = \hat{r}_{l} }} C_{i} (\mathbf{\textbf{r}}^*) - \sum\limits_{\substack{i:i\neq l\\\hat{r}_{i} = \hat{r}_{l} }} C_{i} (\hat{\mathbf{r}}) \right)\\ &&+ \left( \sum\limits_{\substack{i:i\neq l\\ r^*_{i} = r^*_{l} }} C_{i} (\mathbf{\textbf{r}}^*) - \sum\limits_{\substack{i:i\neq l\\ \hat{r}_{i} = r^*_{l} }} C_{i} (\hat{\mathbf{r}}) \right) \\&&+ \left(\sum\limits_{\substack{i:i\neq l,r^*_{i} \neq \hat{r}_{l} \\r^*_{i} \neq r^*_{l} }} C_{i} (\mathbf{\textbf{r}}^*) - \sum\limits_{\substack{i:i\neq l,\hat{r}_{i} = \hat{r}_{l} \\\hat{r}_{i} \neq r^*_{l} }}\hspace*{-6pt}C_{i} (\hat{\mathbf{r}})\hspace*{-3pt}\right).\end{array}$$

STEP 1:

By assumption, we know that $C_{l} (\textbf {r}^*)-C_{l} (\hat {\textbf {r}})<0$.

STEP 2:

We can also deduce that

$$\sum\limits_{\substack{i:i\neq l,r^*_{i} \neq \hat{r}_{l} \\r^*_{i} \neq r^*_{l} }} C_{i} (\mathbf{\textbf{r}}^*) - \sum\limits_{\substack{i:i\neq l,\hat{r}_{i} = \hat{r}_{l} \\\hat{r}_{i} \neq r^*_{l} }} C_{i} (\hat{\mathbf{r}})=0,</p><p class="noindent">$$

because this term represents the sum of the change in the cost values of peers which belong to neither of the two super-peers involved in the transfer of peer l.

STEP 3:

Next we show that

$$\begin{array}{rll} \left( \sum\limits_{\substack{i:i\neq l\\r^*_{i} = \hat{r}_{l} }}{\kern-2pt}C_{i} (\mathbf{\textbf{r}}^*){\kern-2pt}-{\kern-2pt}\sum\limits_{\substack{i:i\neq l\\\hat{r}_{i} = \hat{r}_{l} }}{\kern-2pt}C_{i} (\hat{\mathbf{r}}){\kern-2pt}\right)&+&\left(\sum\limits_{\substack{i:i\neq l\\ r^*_{i} = r^*_{l} }}{\kern-2pt}C_{i} (\mathbf{\textbf{r}}^*) -{\kern-4pt}\right.\left.\sum\limits_{\substack{i:i\neq l\\ \hat{r}_{i} = r^*_{l} }}{\kern-2pt}C_{i} (\hat{\mathbf{r}}){\kern-2pt}\right) \\&<& 0.\end{array}$$

First,

$$ \begin{array}{lll}&&\sum\limits_{\substack{i:i\neq l\\r^*_{i} = \hat{r}_{l} }} C_{i} (\mathbf{\textbf{r}}^*) - \sum\limits_{\substack{i:i\neq l\\\hat{r}_{i} = \hat{r}_{l} }} C_{i} (\hat{\mathbf{r}})\\ &&\qquad =\sum\limits_{\substack{i:i\neq l\\r_{i} ^*=\hat{r}_{l} }} \left(\frac{b_{i} }{w_{r^*_{i} }}\sum\limits_{\substack{j:r^*_{j} =r_{i} ^*\\j\neq i}}b_{j} + \frac{\mu}{2}\sum\limits_{j:r^*_{j} \neq r^*_{i} } c_{ij}\right) \\ &&\qquad\quad- \sum\limits_{\substack{i:i\neq l\\\hat{r}_{i} =\hat{r}_{l} }} \left(\frac{b_{i} }{w_{\hat{r}_{i} }}\sum\limits_{\substack{j:\hat{r}_{j} =\hat{r}_{i} \\j\neq i}}b_{j} + \frac{\mu}{2}\sum\limits_{j:\hat{r}_{j} \neq \hat{r}_{i} } c_{ij}\right) \\ &&\qquad= \sum\limits_{\substack{i:i\neq l\\r_{i} ^*=\hat{r}_{l} }} \left(\frac{b_{i} }{w_{\hat{r}_{l} }}\sum\limits_{\substack{j:r^*_{j} =\hat{r}_{l} \\j\neq i}}b_{j} + \frac{\mu}{2}\sum\limits_{j:r^*_{j} \neq \hat{r}_{l} } c_{ij}\right) \\ &&\qquad\quad-\sum\limits_{\substack{i:i\neq l\\\hat{r}_{i} =\hat{r}_{l} }} \left(\frac{b_{i} }{w_{\hat{r}_{l} }}\sum\limits_{\substack{j:\hat{r}_{j} =\hat{r}_{l} \\j\neq i}}b_{j} + \frac{\mu}{2}\sum\limits_{j:\hat{r}_{j} \neq \hat{r}_{l} } c_{ij}\right). \end{array} $$

Note that the two sets $\{i:i\neq l,\hat {r}_{i} =\hat {r}_{l} \}$ and $\{i:i\neq l,r_{i} ^*=\hat {r}_{l} \}$ are equal because all other assignments except $r_{l}$ are the same in the new assignment vector $\textbf {r}^*$. Thus,

$$\begin{array}{lll}&&\sum\limits_{\substack{i:i\neq l\\r^*_{i} = \hat{r}_{l} }} C_{i} (\mathbf{\textbf{r}}^*) - \sum\limits_{\substack{i:i\neq l\\\hat{r}_{i} = \hat{r}_{l} }} C_{i} (\hat{\mathbf{r}}) \\ &&\qquad=\sum\limits_{\substack{i:i\neq l\\r_{i} ^*=\hat{r}_{l} }} \Biggl( \frac{b_{i} }{w_{\hat{r}_{l} }}\sum\limits_{\substack{j:r^*_{j} =\hat{r}_{l} \\j\neq i}}b_{j} -\frac{b_{i} }{w_{\hat{r}_{l} }}\sum\limits_{\substack{j:\hat{r}_{j} =\hat{r}_{l} \\j\neq i}}b_{j} \\ &&\qquad\quad\Biggl.+ \frac{\mu}{2}\sum\limits_{j:r^*_{j} \neq \hat{r}_{l} } c_{ij} - \frac{\mu}{2}\sum\limits_{j:\hat{r}_{j} \neq \hat{r}_{l} } c_{ij} \Biggr)\\ &&\qquad=\sum\limits_{\substack{i:i\neq l\\r_{i} ^*=\hat{r}_{l} }} \Biggl( \frac{b_{i} }{w_{\hat{r}_{l} }} \Biggl(\sum\limits_{\substack{j:r^*_{j} =\hat{r}_{l} \\j\neq i}}b_{j} - \sum\limits_{\substack{j:\hat{r}_{j} =\hat{r}_{l} \\j\neq i}}b_{j} \Biggr) \\ &&\qquad\quad + \Biggl.\frac{\mu}{2} \Biggl( \sum\limits_{j:r^*_{j} \neq \hat{r}_{l} } c_{ij} - \sum\limits_{j:\hat{r}_{j} \neq \hat{r}_{l} } c_{ij} \Biggr) \Biggr).\end{array}$$

For $i \neq l$, $\{j:r_{j} ^*=\hat {r}_{l} ,j\neq i\}$ represents the set of peers except the lth and ith peer which are currently assigned to $\hat {r}_{l} {}$th super-peer and $\{j:\hat {r}_{j} =\hat {r}_{l} ,j\neq i\}$ represents the set of peers except the lth and ith peer whose new assignment is the $\hat {r}_{l} {}$th super-peer. So, for $i \neq l$, $\{j:r_{j} ^*=\hat {r}_{l} ,j\neq i\} \setminus \{j:\hat {r}_{j} =\hat {r}_{l} ,j\neq i\} = l$. This implies

$$ \sum\limits_{\substack{j:r^*_{j} =\hat{r}_{l} \\j\neq i}}b_{j} - \sum\limits_{\substack{j:\hat{r}_{j} =\hat{r}_{l} \\j\neq i}}b_{j} = -b_{l} . $$

Also, $\{j:r^*_{j} \neq \hat {r}_{l} \} \setminus \{j:\hat {r}_{j} \neq \hat {r}_{l} \} = l$. Thus, we can write

$$ \sum\limits_{j:r^*_{j} \neq \hat{r}_{l} } c_{ij} - \sum\limits_{j:\hat{r}_{j} \neq \hat{r}_{l} } c_{ij} = c_{il}, $$

which implies,

$$ \sum\limits_{\substack{i:i\neq l\\r^*_{i} = \hat{r}_{l} }} C_{i} (\mathbf{\textbf{r}}^*) - \sum\limits_{\substack{i:i\neq l\\\hat{r}_{i} = \hat{r}_{l} }} C_{i} (\hat{\mathbf{r}}) = \sum\limits_{\substack{i:i\neq l\\r_{i} ^*=\hat{r}_{l} }} \Biggl( -b_{l} \left(\frac{b_{i} }{w_{\hat{r}_{l} }}\right) + \frac{\mu}{2} c_{il} \Biggr). $$

Using a similar approach, we can show:

$$\sum\limits_{\substack{i:i\neq l\\ r^*_{i} = r^*_{l} }} C_{i} (\mathbf{\textbf{r}}^*) - \sum\limits_{\substack{i:i\neq l\\ \hat{r}_{i} = r^*_{l} }} C_{i} (\hat{\mathbf{r}}) = \sum\limits_{\substack{i:i\neq l\\r_{i} ^*=r_{l} ^*}} \Biggl( b_{l} \left(\frac{b_{i} }{w_{r_{l} ^*}}\right) - \frac{\mu}{2} c_{il} \Biggr). $$

So,

$$ \begin{array}{lll}&&\Biggl( \sum\limits_{\substack{i:i\neq l\\r^*_{i} = \hat{r}_{l} }} C_{i} (\mathbf{\textbf{r}}^*) - \sum\limits_{\substack{i:i\neq l\\\hat{r}_{i} = \hat{r}_{l} }} C_{i} (\hat{\mathbf{r}}) \Biggr) \\ && \quad\;\;\;+ \Biggl( \sum\limits_{\substack{i:i\neq l\\ r^*_{i} = r^*_{l} }} C_{i} (\mathbf{\textbf{r}}^*) - \sum\limits_{\substack{i:i\neq l\\ \hat{r}_{i} = r^*_{l} }} C_{i} (\hat{\mathbf{r}}) \Biggr) \\ &&\qquad\quad = \sum\limits_{\substack{i:i\neq l\\r_{i} ^*=\hat{r}_{l} }} \Biggr( -b_{l} \left(\frac{b_{i} }{w_{\hat{r}_{l} }}\right) + \frac{\mu}{2} c_{il} \Biggr)\\ && \qquad\qquad\quad+ \sum\limits_{\substack{i:i\neq l\\r_{i} ^*=r_{l} ^*}} \Biggl( b_{l} \left(\frac{b_{i} }{w_{r_{l} ^*}}\right) - \frac{\mu}{2} c_{il} \Biggr).\end{array} $$

(13)

Now consider

$$ \begin{array}{rll} C_{l} (\textbf{r}^*)-C_{l} (\hat{\textbf{r}}) &=& b_{l} \sum\limits_{\substack{i:r^*_{i} =r^*_{l} \\i\neq l}}\frac{b_{i} }{w_{r^*_{l} }}+\frac{\mu}{2}\sum\limits_{\substack{i:r^*_{i} \neq r^*_{l} \\i\neq l}}c_{il} \\&&- b_{l} \sum\limits_{\substack{i:\hat{r}_{i} =\hat{r}_{l} \\i\neq l}}\frac{b_{i} }{w_{\hat{r}_{l} }}-\frac{\mu}{2}\sum\limits_{\substack{i:\hat{r}_{i} \neq \hat{r}_{l} \\i\neq l}}c_{il}.\end{array}$$

We know that,

$$ \begin{array}{lll} &&\sum\limits_{\substack{i:r^*_{i} \neq r^*_{l} \\i\neq l}}c_{il} - \sum\limits_{\substack{i:\hat{r}_{i} \neq \hat{r}_{l} \\i\neq l}}c_{il}= \sum\limits_{\substack{r^*_{i} = \hat{r}_{l} \\i\neq l}}c_{il} + \sum\limits_{\substack{i:r^*_{i} \neq r^*_{l} \\\substack{r^*_{i} \neq \hat{r}_{l} \\i\neq l}}}c_{il} - \\ &&\qquad\qquad\qquad\qquad\;\sum\limits_{\substack{\hat{r}_{i} = r^*_{l} \\i\neq l}}c_{il} - \sum\limits_{\substack{i:\hat{r}_{i} \neq \hat{r}_{l} \\ \substack{\hat{r}_{i} \neq r^*_{l} \\i\neq l}}}c_{il}.\end{array}$$

(14)

However,

$$ \sum\limits_{\substack{i:r^*_{i} \neq r^*_{l} \\\substack{r^*_{i} \neq \hat{r}_{l} \\i\neq l}}}c_{il} = \sum\limits_{\substack{i:\hat{r}_{i} \neq \hat{r}_{l} \\ \substack{\hat{r}_{i} \neq r^*_{l} \\i\neq l}}}c_{il}. $$

Thus we see:

$$ \begin{array}{lll} &&\sum\limits_{\substack{i:r^*_{i} \neq r^*_{l} \\i\neq l}}c_{il} - \sum\limits_{\substack{i:\hat{r}_{i} \neq \hat{r}_{l} \\i\neq l}}c_{il}= \sum\limits_{\substack{r^*_{i} = \hat{r}_{l} \\i\neq l}}c_{il} - \sum\limits_{\substack{\hat{r}_{i} = r^*_{l} \\i\neq l}}c_{il}\\&&\Rightarrow C_{l} (\textbf{r}^*)-C_{l} (\hat{\textbf{r}})= -b_{l} \sum\limits_{\substack{i:\hat{r}_{i} =\hat{r}_{l} \\i\neq l}}\frac{b_{i} }{w_{\hat{r}_{l} }}+\frac{\mu}{2}\sum\limits_{\substack{r^*_{i} = \hat{r}_{l} \\i\neq l}}c_{il} \\ &&\qquad +b_{l} \sum\limits_{\substack{i:r^*_{i} =r^*_{l} \\i\neq l}}\frac{b_{i} }{w_{r^*_{l} }}-\frac{\mu}{2}\sum\limits_{\substack{\hat{r}_{i} = r^*_{l} \\i\neq l}}c_{il}.\end{array} $$

But we also know that $\{i:\hat {r}_{i} =\hat {r}_{l} ,i\neq l\} = \{i:r^*_{i} =\hat {r}_{l} ,i\neq l\}$ and $\{\hat {r}_{i} = r^*_{l} ,i\neq l\}=\{r^*_{i} = r^*_{l} ,i\neq l\}$. So,

$$ \begin{array}{lll} &&C_{l} (\textbf{r}^*)-C_{l} (\hat{\textbf{r}}) =-b_{l} \sum\limits_{\substack{i:r^*_{i} =\hat{r}_{l} \\i\neq l}}\frac{b_{i} }{w_{\hat{r}_{l} }}+\frac{\mu}{2}\sum\limits_{\substack{r^*_{i} = \hat{r}_{l} \\i\neq l}}c_{il} \\&&\qquad +b_{l} \sum\limits_{\substack{i:r^*_{i} =r^*_{l} \\i\neq l}}\frac{b_{i} }{w_{r^*_{l} }}-\frac{\mu}{2}\sum\limits_{\substack{r^*_{i} = r^*_{l} \\i\neq l}}c_{il}\\&&=\sum\limits_{\substack{i:i\neq l\\r_{i} ^*=\hat{r}_{l} }} \Biggl( -b_{l} \left(\frac{b_{i} }{w_{\hat{r}_{l} }}\right) + \frac{\mu}{2} c_{il} \Biggr)+\sum\limits_{\substack{i:i\neq l\\r_{i} ^*=r_{l} ^*}} \Biggl( b_{l} \left(\frac{b_{i} }{w_{r_{l} ^*}}\right) - \frac{\mu}{2} c_{il} \Biggr)\\ &&=\Biggl( \sum\limits_{\substack{i:i\neq l\\r^*_{i} = \hat{r}_{l} }} C_{i} (\mathbf{\textbf{r}}^*) - \sum\limits_{\substack{i:i\neq l\\\hat{r}_{i} = \hat{r}_{l} }} C_{i} (\hat{\mathbf{r}}) \Biggr) \\&&\qquad + \Biggl( \sum\limits_{\substack{i:i\neq l\\ r^*_{i} = r^*_{l} }} C_{i} (\mathbf{\textbf{r}}^*) - \sum\limits_{\substack{i:i\neq l\\ \hat{r}_{i} = r^*_{l} }} C_{i} (\hat{\mathbf{r}}) \Biggr).\end{array} $$

(15)

The last step follows from Eq. 13. Thus we have shown:

$$\begin{array}{lll} &&\Bigl( \sum\limits_{\substack{i:i\neq l\\r^*_{i} = \hat{r}_{l} }} C_{i} (\mathbf{\textbf{r}}^*) - \sum\limits_{\substack{i:i\neq l\\\hat{r}_{i} = \hat{r}_{l} }} C_{i} (\hat{\mathbf{r}}) \Bigr)+ \Bigl( \sum\limits_{\substack{i:i\neq l\\ r^*_{i} = r^*_{l} }} C_{i} (\mathbf{\textbf{r}}^*) - \sum\limits_{\substack{i:i\neq l\\ \hat{r}_{i} = r^*_{l} }} C_{i} (\hat{\mathbf{r}}) \Bigr) \\ &&\qquad = C_{l} (\textbf{r}^*)-C_{l} (\hat{\textbf{r}}) < 0\end{array} $$

(16)

This completes the proof for STEP 3.

Hence using Steps $1, 2$ and 3, $C_{0}(\textbf {r}^*)-C_{0}(\hat {\textbf {r}})= 2\bigl (C_{l} (\textbf {r}^*)-C_{l} (\hat {\textbf {r}})\bigr )< 0$. This contradicts our assumption that $\hat {\textbf {r}}$ is a globally optimal solution of $C_{0}$. Hence $\hat {\textbf {r}}$ is also a Nash equilibrium in pure strategies. □