A Stochastic Differential Game Theoretic Study of Multipath Routing in Heterogeneous Wireless Networks

Abstract

In heterogeneous wireless networks (HWNs), paths constituting multipath routing are characterized by selfish rationality. Each path’s intentions of pursuing individual profits may cause unreasonable competition for limited wireless resources, which leads to the unreliable data transport problem. Therefore, multipath routing optimization is a challenge issue in HWNs. This paper provides a novel approach to study this issue by employing game theory. By taking the utility maximization as the design goal, limited bandwidth resources as the constraint and path reliability as the key metric, a noncooperative stochastic differential game model is constructed for HWNs. With the feedback Nash equilibrium solution, an optimal multipath routing strategy is obtained. Theoretical derivations and simulation results verify the validity of the method present in this paper.

Appendices

Appendix 1: Proof of Corollary 1

Proof

Maximizing the right-hand-side of (10) for path-$i$, and arranging the equation we get

$$\begin{aligned} \sum _{j=1,j\ne i}^n {\phi _j^*\left( {t,x} \right) } \hbox {+2}\phi _{_i }^*\left( {t,x} \right) =\xi ^{-1}\left\{ {\lambda -\delta q_i {+}\exp [r(t-t_0 )]p_i V_x^i } \right\} \end{aligned}$$

(18)

For $i = 1,2,\ldots ,n$, there is an system of linear equations

$$\begin{aligned} \left\{ {{\begin{array}{l} {2\phi _1^*\left( {t,x} \right) {+}\phi _2^*\left( {t,x} \right) {+}\cdots {+}\phi _n^*\left( {t,x} \right) = \xi ^{-1}\left\{ {\lambda -\delta q_1 +\exp [r(t-t_0 )]p_1 V_x^1 } \right\} } \\ {\phi _1^*\left( {t,x} \right) \hbox {+2}\phi _2^*\left( {t,x} \right) {+}\cdots {+}\phi _n^*\left( {t,x} \right) = \xi ^{-1}\left\{ {\lambda -\delta q_2 +\exp [r(t-t_0 )]p_2 V_x^2 } \right\} } \\ \vdots \\ {\phi _1^*\left( {t,x} \right) {+}\phi _2^*\left( {t,x} \right) {+}\cdots \hbox {+2}\phi _n^*\left( {t,x} \right) = \xi ^{-1}\left\{ {\lambda -\delta q_n +\exp [r(t-t_0 )]p_n V_x^n } \right\} } \\ \end{array} }} \right. \end{aligned}$$

(19)

Summing over the right-hand-side and the left-hand-side respectively in (18), we get

$$\begin{aligned}&\phi _1^*\left( {t,x} \right) +\phi _2^*\left( {t,x} \right) +\cdots +\phi _n^*\left( {t,x} \right) \nonumber \\&\quad = \ \left[ {\xi \left( {n+1} \right) } \right] ^{-1}\left\{ {n\lambda -\delta \sum _{j=1}^n {q_j } +\exp \left[ {r\left( {t-t_0 } \right) } \right] \sum _{j=1}^n {p_j V_x^j } } \right\} \end{aligned}$$

(20)

There, we have two methods to obtain the solution of $\phi _i^*\left( {t,x} \right)$. $\square$

Solution 1

Substituting (20) into (18), we get

$$\begin{aligned}&\left[ {\xi \left( {n+1} \right) } \right] ^{-1}\sum _{j=1}^n {\left\{ {\lambda -\delta q_j -\exp [r(t-t_0 )]p_j V_x^j } \right\} } +\phi _i^*\left( {t,x} \right) \nonumber \\&\quad = \ \xi ^{-1}\left\{ {\lambda -\delta q_i {+}\exp [r(t-t_0 )]p_i V_x^i } \right\} \end{aligned}$$

(21)

Arranging (21), then

$$\begin{aligned} \phi _i^*\left( {t,x} \right)&= \left[ {\xi \left( {n+1} \right) } \right] ^{-1}\left\{ \lambda +\delta \left( {\sum _{j=1,j\ne i}^n {q_j } {-}nq_i } \right) \right. \\&\left. -\exp \left[ {r\left( {t-t_0 } \right) } \right] \left[ {\sum _{j=1,j\ne i}^n {p_j V_x^j } {-}np_i V_x^i } \right] \right\} \end{aligned}$$

Solution 2

Letting (18) minus (20), we directly get

$$\begin{aligned} \phi _i^*\left( {t,x} \right)&= \left[ {\xi \left( {n+1} \right) } \right] ^{-1}\left\{ \lambda +\delta \left( {\sum _{j=1,j\ne i}^n {q_j } {-}nq_i } \right) \right. \\&\left. -\exp \left[ {r\left( {t-t_0 } \right) } \right] \left[ {\sum _{j=1,j\ne i}^n {p_j V_x^j } {-}np_i V_x^i } \right] \right\} \end{aligned}$$

This completes the proof of Corollary 1.

Appendix 2: Proof of Corollary 2

Proof

From (14), we get

$$\begin{aligned} \left\{ {{\begin{array}{l} V_x^i (t,x) = \hbox {exp}\left[ {-r\left( {t-t_0 } \right) } \right] \Gamma (t) \\ V_{xx}^i (t,x) = 0 \\ V_t^i (t,x) = \hbox {exp}\left[ {-r\left( {t-t_0 } \right) } \right] \left\{ {-r\left[ {\Gamma (t)x+\Lambda (t)} \right] +\left[ {\dot{\Gamma }(t)x+\dot{\Lambda }(t)} \right] } \right\} \\ \end{array} }} \right. \end{aligned}$$

(22)

Substituting (22) into (11) produces

$$\begin{aligned}&-r\left[ {\Gamma (t)x+\Lambda (t)} \right] +\left[ {\dot{\Gamma }(t)x+\dot{\Lambda }(t)} \right] \nonumber \\&\quad = \ \left[ {\lambda \phi _i^*\left( {t,x} \right) -\xi \left( {\sum _{j=1}^n {\phi _j^*\left( {t,x} \right) } } \right) \phi _i^*\left( {t,x} \right) -\delta q_i \phi _i^*\left( {t,x} \right) } \right] \nonumber \\&\qquad +\Gamma (t)\left[ {ax-\left( {\sum _{j=1}^n {\phi _j^*\left( {t,x} \right) } } \right) } \right] \nonumber \\&\quad = \ \left[ {\left( {\lambda -\delta q_i } \right) \phi _i^*\left( {t,x} \right) -\xi \left( {\sum _{j=1}^n {\phi _j^*\left( {t,x} \right) } } \right) \phi _i^*\left( {t,x} \right) } \right] \nonumber \\&\qquad +\Gamma (t)\left[ {ax-\left( {\sum _{j=1}^n {\phi _j^*\left( {t,x} \right) } } \right) } \right] \end{aligned}$$

(23)

Meanwhile, substituting (22) into (13) produces

$$\begin{aligned} \phi _i^*\left( {t,x} \right) =f_i +\exp \left[ {r\left( {t-t_0 } \right) } \right] \sum _{i=1}^n {g_i V_x^i } = f_i {+}\Gamma (t)\sum _{i=1}^n {g_i } \end{aligned}$$

(24)

Thus

$$\begin{aligned} \dot{\Gamma }(t)x+\dot{\Lambda }(t)&= r\left[ {\Gamma (t)x+\Lambda (t)} \right] +\left[ {\lambda {-}\delta q_i -\xi \left( {\sum _{j=1}^n {\phi _j^*\left( {t,x} \right) } } \right) } \right] \left( {f_i {+}\Gamma (t)\sum _{i=1}^n {g_i } } \right) \nonumber \\&+\, \Gamma (t)\left[ {ax-\left( {\sum _{j=1}^n {\phi _j^*\left( {t,x} \right) } } \right) } \right] \nonumber \\&= \left( {r+a} \right) \Gamma (t)x+r\Lambda (t)+\left[ {\lambda {-}\delta q_i -\xi \left( {\sum _{i=1}^n {\left( {f_i {+}\Gamma (t)\sum _{i=1}^n {g_i } } \right) } } \right) } \right] \nonumber \\&\left( {f_i +\Gamma (t)\sum _{i=1}^n {g_i } } \right) -\Gamma (t)\sum _{i=1}^n {\left( {f_i {+}\Gamma (t)\sum _{i=1}^n {g_i } } \right) }\nonumber \\&= \left( {r+a} \right) \Gamma (t)x+r\Lambda (t)+\left( {\lambda -\delta q_i } \right) \left( {f_i {+}\Gamma (t)\sum _{i=1}^n {g_i } } \right) \nonumber \\&- \, \left[ {\xi \left( {f_i {+}\Gamma (t)\sum _{i=1}^n {g_i } } \right) {+}\Gamma (t)} \right] \sum _{i=1}^n {\left( {f_i {+}\Gamma (t)\sum _{i=1}^n {g_i } } \right) } \end{aligned}$$

(25)

Let $\Theta (t)=\left( {\lambda {-}\delta q_i } \right) \left( {f_i {+}\Gamma (t)\sum _{i=1}^n {g_i } } \right) -\left[ {\xi \left( {f_i {+}\Gamma (t)\sum _{i=1}^n {g_i } } \right) {+}\Gamma (t)} \right] \sum _{i=1}^n \Big ( f_i {+}\Gamma (t)\sum _{i=1}^n {g_i } \Big ),$

Then

$$\begin{aligned} \dot{\Gamma }(t)&= \left( {r+a} \right) \Gamma (t), \Gamma (T)=\mu ,\\ \dot{\Lambda }(t)&= r\Lambda (t)+\Theta (t), \Lambda (T)=-\mu \bar{{x}}_i . \end{aligned}$$

This completes the proof of Corollary 2. $\square$

Appendix 3: Derivation of (16)

Derivation Equation (15) is constituted by

$$\begin{aligned} \dot{\Gamma }(t)&= \left( {r+a} \right) \Gamma (t), \end{aligned}$$

(15-1)

$$\begin{aligned} \Gamma (T)&=\mu , \end{aligned}$$

(15-2)

$$\begin{aligned} \dot{\Lambda }(t)&= r\Lambda (t)+\Theta (t), \end{aligned}$$

(15-3)

$$\begin{aligned} \Lambda (T)&=-\mu \bar{{x}}_i , \end{aligned}$$

(15-4)

and

$$\begin{aligned} \Theta (t)=\left( {\lambda {-}\delta q_i } \right) \left( {f_i {+}\Gamma (t)\sum _{i=1}^n {g_i } } \right) \!-\!\left[ {\xi \left( {f_i {+}\Gamma (t)\sum _{i=1}^n {g_i } } \right) {+}\Gamma (t)} \right] \sum _{i=1}^n {\left( {f_i {+}\Gamma (t)\sum _{i=1}^n {g_i } } \right) }. \end{aligned}$$

(15-5)

It is noted that (15-1) is a homogeneous linear differential equation, while (15-3) is a nonhomogeneous linear differential equation, and both of them have their general solutions. The derivation of (16) is related to the solution process of linear differential equation. The derivation process is detailed as following:

Separate the variable of (15-1), we get

$$\begin{aligned} \frac{d\Gamma (t)}{\Gamma (t)} = \left( {r+a} \right) t. \end{aligned}$$

(26)

Integrate both sides of (26), we get

$$\begin{aligned} \ln \Gamma (t)= -\int _{t_0 }^t {\left( {r+a} \right) t} +\ln \overline{{\Gamma }}, \quad \overline{{\Gamma }}\hbox { is a constant. } \end{aligned}$$

(27)

Then,

$$\begin{aligned} \Gamma (t)&=\overline{{\Gamma }}\cdot \hbox {exp}\left[ {\int _{t_0 }^t {\left( {r+a} \right) t} } \right] \\&=\hbox {exp}\left[ {\left( {r+a} \right) \left( {t-t_0 } \right) } \right] \overline{{\Gamma }}. \end{aligned}$$

(16-1)

When $t=T$,

$$\begin{aligned} \Gamma (T) = \overline{{\Gamma }}\cdot \hbox {exp}\left[ {\left( {r+a} \right) \left( {T-t_0 } \right) } \right] . \end{aligned}$$

(28)

Combining (28) and (15-2), we get

$$\begin{aligned} \overline{{\Gamma }}=\mu \hbox {exp}\left[ {-\left( {r+a} \right) \left( {T-t_0 } \right) } \right] . \end{aligned}$$

(16-2)

Let

$$\begin{aligned} \Lambda (t)&= Q(t)\cdot \exp \left[ {\int _{t_0 }^t {rds} } \right] \nonumber \\&= \, Q(t)\cdot \exp \left[ {r\left( {t-t_0 } \right) } \right] \end{aligned}$$

(29)

be the solution of the nonhomogeneous linear differential equation (15-3).

Substitute $\dot{\Lambda }(t)$ and $\Lambda (t)$ into (15-3), then

$$\begin{aligned} \dot{Q}(t)\cdot \exp \left[ {r\left( {s-t_0 } \right) } \right] {+}Q(t)\cdot \exp \left[ {r\left( {t-t_0 } \right) } \right] \cdot r{=}r\cdot Q(t)\cdot \exp \left[ {r\left( {t-t_0 } \right) } \right] +\Theta (t). \end{aligned}$$

(30)

Arrange (30), we get

$$\begin{aligned} \dot{Q}(t)=\Theta (t)\cdot \exp \left[ {-r\left( {s-t_0 } \right) } \right] . \end{aligned}$$

(31)

Integrate both sides of (31), we get

$$\begin{aligned} Q(t)=\int _{t_0 }^t {\Theta (s)\cdot \exp \left[ {-r\left( {s-t_0 } \right) } \right] ds} +\bar{{\Lambda }}, \quad \bar{{\Lambda }}\hbox { is a constant. } \end{aligned}$$

(32)

Substitute (32) into (29), we get

$$\begin{aligned} \Lambda (t){=}\exp \left[ {r\left( {t-t_0 } \right) } \right] \left( {\int _{t_0 }^t {\Theta (s)\cdot \exp \left[ {-r\left( {s-t_0 } \right) } \right] ds+\bar{{\Lambda }}} } \right) . \end{aligned}$$

(16-3)

When $t=T$,

$$\begin{aligned} \Lambda (T){=}\left( {\int _{t_0 }^T {\Theta (t)\cdot \exp \left[ {-r\left( {s-t_0 } \right) } \right] ds+\bar{{\Lambda }}} } \right) \cdot \exp \left[ {r\left( {T-t_0 } \right) } \right] . \end{aligned}$$

(33)

Combining (33) and (15-4), we get

$$\begin{aligned} \bar{{\Lambda }}{=}-\mu \bar{{x}}_i \hbox {exp}\left[ {-r\left( {T-t_0 } \right) } \right] -\int _{t_0 }^T {\Theta (s)\hbox {exp}\left[ {-r\left( {s-t_0 } \right) } \right] ds} . \end{aligned}$$

(16-4)

Equations (16-1), (16-2), (16-3) and (16-4) constitute (16).

This completes the derivation of (16).

Appendix 4: Proof of Corollary 3

Proof

Solving the linear differential equation (9), we get

$$\begin{aligned} x^{*}(s)=\exp \left[ {\int _{t_0 }^t {ads} +\int _{t_0 }^t {\sigma dz(s)} } \right] \times \left\{ {x_0 +\sum _{i=1}^n {\left( {p_i \phi _i^*\left( {t,x} \right) } \right) } \left[ {\int _{t_0 }^t {ads} +\int _{t_0 }^t {\sigma dz(s)} } \right] } \right\} . \end{aligned}$$

(34)

Substituting the optimal load-control strategy (24) into (34) produces

$$\begin{aligned} x^{*}(s)&= \exp \left[ {\int _{t_0 }^t {ads} +\int _{t_0 }^t {\sigma dz(s)} } \right] \\&\times \left\{ {x_0 +\sum _{i=1}^n {p_i \left( {f_i {+}\Gamma (t)\sum _{i=1}^n {g_i } } \right) } \exp \left[ {-\left( {\int _{t_0 }^t {ads} +\int _{t_0 }^t {\sigma dz(s)} } \right) } \right] } \right\} . \end{aligned}$$

This completes the proof of Corollary 3. $\square$