Dynamic Channel Allocation in Wireless Networks Using Adaptive Learning Automata

Abstract

Single-channel based wireless networks have limited bandwidth and throughput and the bandwidth utilization decreases with increased number of users. To mitigate this problem, simultaneous transmission on multiple channels is considered as an option. In this paper, we propose a distributed dynamic channel allocation scheme using adaptive learning automata for wireless networks whose nodes are equipped with single-radio interfaces. The proposed scheme, Adaptive Pursuit learning automata runs periodically on the nodes, and adaptively finds the suitable channel allocation in order to attain a desired performance. A novel performance index, which takes into account the throughput and the energy consumption, is considered. The proposed learning scheme adapts the probabilities of selecting each channel as a function of the error in the performance index at each step. The extensive simulation results in static and mobile environments provide that the proposed channel allocation schemes in the multiple channel wireless networks significantly improves the throughput, drop rate, energy consumption per packet and fairness index—compared to the 802.11 single-channel, and 802.11 with randomly allocated multiple channels. Also, it was demonstrated that the Adaptive Pursuit Reward-Only (PRO) scheme guarantees updating the probability of the channel selection for all the links—even the links whose current channel allocations do not provide a satisfactory performance—thereby reducing the frequent channel switching of the links that cannot achieve the desired performance.

Appendix: Proof of Convergence

In Sect. 2.2, the channel allocation algorithms were presented. In this section, the proofs of convergence of the algorithms are presented. The proofs follow the general method used in [19].

1.1 Proof of Convergence of the Adaptive PRI Algorithm

Theorem 1 establishes that for each node that is running the algorithm, if after a certain time, the channel allocation results in a greater performance for one channel compared to the other channels, the probability of selecting that channel tends to 1. Theorem 2 establishes that for each node and each channel, there exists a time that the channel has been selected by the node for at least M times. This guarantees having the average throughput, delay and consumed energy values, which are required for the performance evaluation.

Theorem 1

Suppose there exists an index m _i and a time instant k ₀  < ∞ such that $\hat{\phi }_{i}^{{m_{i} }} \left( k \right) > \hat{\phi }_{i}^{j} \left( k \right)$ for all j such that j ≠ m _i and all k ≥ k ₀ . Then there exists γ ₀ and λ ₀ such that for all resolution parameters (γ < γ ₀ , λ < λ ₀ ), $p_{i}^{{m_{i} }} \left( k \right) \to 1$ with probability 1 as $k \to \infty .$

Proof From the definition for Discrete PRI, we know that if m _i satisfies

$m_{i} = { \arg }\;{ \max }_{j} \hat{\phi }_{i}^{j} \left( k \right),$ where $\hat{\phi }_{i}^{{m_{i} }} \left( k \right) = { \max }_{j} \hat{\phi }_{i}^{j} \left( k \right),$ then $\hat{\phi }_{i}^{{m_{i} }} \left( k \right) > \hat{\phi }_{i}^{j} \left( k \right)$ for all j ≠ m _i and all k ≥ k ₀.

Therefore, for all k > k ₀,

$$ p_{i}^{{m_{i} }} \left( {k + 1} \right) = \left\{ {\begin{array}{*{20}c} {1 - \sum\nolimits_{{j = 1,j \ne m_{i} }}^{N} {\left( {p_{i}^{j} \left( k \right) - \theta \left( k \right)} \right),} } & {{\text{if}}\;\beta_{i}^{l} \left( k \right) = 0\;\left( {{\text{w}} . {\text{p}} .\;\zeta_{i}^{{m_{i} }} \left( k \right)} \right)} \\ {p_{i}^{{m_{i} }} \left( k \right)} & {{\text{if}}\;\beta_{i}^{l} \left( k \right) = 1\;\left( {{\text{w}} . {\text{p}} .\;1 - \zeta_{i}^{{m_{i} }} \left( k \right)} \right)} \\ \end{array} } \right. $$

(7)

If $p_{i}^{{m_{i} }} \left( k \right) = 1,$ then the “pursuit” property of the algorithm trivially proves the result.

Assuming that the algorithm has not yet converged to the m _i th channel, there exists at least one nonzero component of P _i (k), p ^q _i (k), with q ≠ m _i . Therefore, we can write

$$ p_{i}^{q} \left( {k + 1} \right) = p_{i}^{q} \left( k \right) - \theta \left( k \right) < p_{i}^{q} \left( k \right). $$

(8)

Since P _i (k) is a probability vector, $\sum\nolimits_{j = 1}^{N} {p_{i}^{j} \left( k \right)} = 1,$ and $p_{i}^{{m_{i} }} \left( k \right) = 1 - \sum\nolimits_{{j = 1,j \ne m_{i} }}^{N} {p_{i}^{j} \left( k \right)} .$ Therefore,

$$ 1 - \sum\limits_{{j = 1,j \ne m_{i} }}^{N} {\left( {p_{i}^{j} \left( k \right) - \theta \left( k \right)} \right)} > p_{i}^{{m_{i} }} \left( k \right). $$

(9)

As long as there is at least one nonzero component, p ^q _i (k) (where q ≠ m _i ), it is clear that we can decrement p ^q _i (k) and increment $p_{i}^{{m_{i} }} \left( k \right)$ by at least θ(k). Hence, $p_{i}^{{m_{i} }} \left( {k + 1} \right) = p_{i}^{{m_{i} }} \left( k \right) + c\left( k \right) \cdot \theta \left( k \right),$ where c(k) · θ(k) is an integral multiple of θ(k), and 0 < c(k) < N, and

$$ \theta \left( k \right) = \left\{ {\begin{array}{*{20}c} {\gamma \cdot {{\left| {\Updelta \left( k \right)} \right|} \mathord{\left/ {\vphantom {{\left| {\Updelta \left( k \right)} \right|} {\phi^{ * } ,}}} \right. \kern-\nulldelimiterspace} {\phi^{ * } ,}}} & {{\text{if}}\; - \delta < {{\Updelta \left( k \right)} \mathord{\left/ {\vphantom {{\Updelta \left( k \right)} {\phi^{ * } }}} \right. \kern-\nulldelimiterspace} {\phi^{ * } }}} \\ {\lambda \cdot {{\left| {\Updelta \left( k \right)} \right|} \mathord{\left/ {\vphantom {{\left| {\Updelta \left( k \right)} \right|} {\phi^{ * } }}} \right. \kern-\nulldelimiterspace} {\phi^{ * } }}} & {\text{otherwise}} \\ \end{array} } \right. $$

(10)

Therefore we can express the expected value of $p_{i}^{{m_{i} }} \left( {k + 1} \right)$ conditioned on the current state of the channel, Q(k), (Q(k) = {P _i (k), ϕ _i (k)}) as follows

$$ \begin{aligned} E\left[ {p_{i}^{{m_{i} }} \left( {k + 1} \right)|{\mathbf{Q}}\left( k \right),p_{i}^{{m_{i} }} \left( k \right) \ne 1} \right] & = \zeta_{i}^{{m_{i} }} \left( k \right) \cdot \left[ {p_{i}^{{m_{i} }} \left( k \right) + c\left( k \right) \cdot \theta \left( k \right)} \right] + \left( {1 - \zeta_{i}^{{m_{i} }} \left( k \right)} \right) \cdot p_{i}^{{m_{i} }} \left( k \right) \\ & = p_{i}^{{m_{i} }} \left( k \right) + \zeta_{i}^{{m_{i} }} \left( k \right) \cdot c\left( k \right) \cdot \theta \left( k \right) \\ \end{aligned} $$

(11)

Since all the previous terms have an upperbound of unity, $E\left[ {p_{i}^{{m_{i} }} \left( {k + 1} \right)|Q\left( k \right),p_{i}^{{m_{i} }} \left( k \right) \ne 1} \right]$ is also bounded,

$$ \mathop { \sup }\limits_{k \ge 0} \;E\left[ {p_{i}^{{m_{i} }} \left( {k + 1} \right)|{\mathbf{Q}}\left( k \right),p_{i}^{{m_{i} }} \left( k \right) \ne 1} \right] < \infty . $$

(12)

Thus we can write $E\left[ {p_{i}^{{m_{i} }} \left( {k + 1} \right) - p_{i}^{{m_{i} }} \left( k \right)|{\mathbf{Q}}\left( k \right)} \right] = \zeta_{i}^{{m_{i} }} \left( k \right) \cdot c\left( k \right) \cdot \theta \left( k \right) \ge 0,\;{\text{ for}}\;{\text{all}}\;k \ge k_{0}$ implying that $p_{i}^{{m_{i} }} \left( k \right)$ is submartingale. By submartingale convergence theorem, the sequence $\{ p_{i}^{{m_{i} }} \left( k \right)\}_{{k \ge k_{0} }}$ converges.

Therefore, $E\left[ {p_{i}^{{m_{i} }} \left( {k + 1} \right) - p_{i}^{{m_{i} }} \left( k \right)|{\mathbf{Q}}\left( k \right)} \right] \to 0\;{\text{w}} . {\text{p}}.1,\;{\text{as}}\;k \to \infty .$

This implies that $\zeta_{i}^{{m_{i} }} \left( k \right) \cdot c\left( k \right) \cdot \theta \left( k \right) \to 0\;{\text{w}} . {\text{p}}.1.$ This in turn implies that $c\left( k \right) \to 0\;{\text{w}} . {\text{p}}.1$ $\, \left( {\theta \left( k \right) \to 0\;{\text{w}} . {\text{p}}.1} \right),$ which means there is no nonzero element in P _i (k) except for $p_{i}^{{m_{i} }} \left( k \right)$ (or Δ(k) → 0). Consequently, $\sum\nolimits_{{j = 1,j \ne m_{i} }}^{N} {p_{i}^{j} \left( k \right)} \to 0\;{\text{w}} . {\text{p}}.1$ and $p_{i}^{{m_{i} }} \left( k \right) = 1 - \sum\nolimits_{{j = 1,j \ne m_{i} }}^{N} {p_{i}^{j} \left( k \right)} \to 1\;{\text{w}} . {\text{p}}.1.$□

Theorem 2

For each node i and channel j, assume p ^j _i (0) ≠ 0. Then for any given constant δ ₀  > 0 and M < ∞, there exists γ ₀  < ∞, λ ₀  < ∞ and k ₀  < ∞ such that under the Discrete PRI algorithm, for all learning parameters γ < γ ₀ and λ < λ ₀ and all time k > k ₀ :

$$ { \Pr }\left\{ {{\text{each}}\;{\text{channel}}\;{\text{chosen}}\;{\text{by}}\;{\text{node}}\;i\;{\text{more}}\;{\text{than}}\;M\;{\text{times}}\;{\text{at}}\;{\text{time}}\;k} \right\} \ge 1 - \delta_{0} . $$

Proof

Define the random variable Y ^j _i (k) as the number of times that channel j was chosen by node i up to time k, then we must prove that ${ \Pr }\left\{ {Y_{i}^{j} \left( k \right) > M} \right\} \ge 1 - \delta_{0} .$ This is equivalent to proving

$$ { \Pr }\left\{ {Y_{i}^{j} \left( k \right) \le M} \right\} \le \delta_{0} . $$

(13)

The events Y ^j _i (k) = q and Y ^j _i (k) = s are mutually exclusive for q ≠ s, so we can rewrite Eq. (13) as

$$ \sum\limits_{q = 1}^{M} {{ \Pr }\{ Y_{i}^{j} \left( k \right) = q\} } \le \delta_{0} . . $$

(14)

For any iteration of the algorithm, Pr{choosing channel j} ≤ 1. Also the magnitude by which any channel selection probability can decrease in any iteration is bounded by ${{\gamma \cdot \left| {\Updelta \left( k \right)} \right|} \mathord{\left/ {\vphantom {{\gamma \cdot \left| {\Updelta \left( k \right)} \right|} {\phi^{ * } }}} \right. \kern-\nulldelimiterspace} {\phi^{ * } }}$ (or ${{\lambda \cdot \left| {\Updelta \left( k \right)} \right|} \mathord{\left/ {\vphantom {{\lambda \cdot \left| {\Updelta \left( k \right)} \right|} {\phi^{ * } }}} \right. \kern-\nulldelimiterspace} {\phi^{ * } }}$), where $\Updelta \left( k \right) < \Updelta$ for all k. During any of the first k iterations of the algorithm:

$$ { \Pr }\left\{ {{\text{channel}}\;j\;{\text{is}}\;{\text{not}}\;{\text{chosen}}\;{\text{by}}\;{\text{node}}\;i} \right\} \le \left( {1 - p_{i}^{j} \left( 0 \right) + k \cdot \gamma \cdot {{\left| \Updelta \right|} \mathord{\left/ {\vphantom {{\left| \Updelta \right|} {\phi^{ * } }}} \right. \kern-\nulldelimiterspace} {\phi^{ * } }}} \right). $$

(15)

Using these upper bounds, the probability that channel j is chosen at most M times among k choices, has the following upper bound

$$ { \Pr }\left\{ {Y_{i}^{j} \left( k \right) \le M} \right\} \le \sum\limits_{l = 1}^{M} {C\left( {k,l} \right)\left( 1 \right)^{l} \left( {1 - p_{i}^{j} \left( 0 \right) + k \cdot \gamma \cdot {{\left| \Updelta \right|} \mathord{\left/ {\vphantom {{\left| \Updelta \right|} {\phi^{ * } }}} \right. \kern-\nulldelimiterspace} {\phi^{ * } }}} \right)^{k - l} } $$

(16)

In order to make a sum of M terms less than δ ₀, it is sufficient to make each term less than ${{\delta_{0} } \mathord{\left/ {\vphantom {{\delta_{0} } M}} \right. \kern-\nulldelimiterspace} M}.$ Consider an arbitrary term, l = m. We must show that

$$ C\left( {k,m} \right)\left( 1 \right)^{m} \left( {1 - p_{i}^{j} \left( 0 \right) + k \cdot \gamma \cdot {{\left| \Updelta \right|} \mathord{\left/ {\vphantom {{\left| \Updelta \right|} {\phi^{ * } }}} \right. \kern-\nulldelimiterspace} {\phi^{ * } }}} \right)^{k - m} < {{\delta_{0} } \mathord{\left/ {\vphantom {{\delta_{0} } M}} \right. \kern-\nulldelimiterspace} M},\quad {\text{or}}\quad M \cdot C\left( {k,m} \right)\left( 1 \right)^{m} \left( {1 - p_{i}^{j} \left( 0 \right) + k \cdot \gamma \cdot {{\left| \Updelta \right|} \mathord{\left/ {\vphantom {{\left| \Updelta \right|} {\phi^{ * } }}} \right. \kern-\nulldelimiterspace} {\phi^{ * } }}} \right)^{k - m} < \delta_{0} . $$

(17)

Knowing that $C\left( {k,m} \right) \le k^{m} ,$ we have to prove that $M \cdot k^{m} \left( {1 - p_{i}^{j} \left( 0 \right) + k \cdot \gamma \cdot {{\left| \Updelta \right|} \mathord{\left/ {\vphantom {{\left| \Updelta \right|} {\phi^{ * } }}} \right. \kern-\nulldelimiterspace} {\phi^{ * } }}} \right)^{k - m} \le \delta_{0} .$

Now in order to get the L.H.S. of this term to be less than δ ₀ as k increases, $\left( {1 - p_{i}^{j} \left( 0 \right) + k \cdot \gamma \cdot {{\left| \Updelta \right|} \mathord{\left/ {\vphantom {{\left| \Updelta \right|} {\phi^{ * } }}} \right. \kern-\nulldelimiterspace} {\phi^{ * } }}} \right)$ must be strictly less than unity. In order to guarantee this, we bound the value of γ with respect to k in such a way that $\left( {1 - p_{i}^{j} \left( 0 \right) + k \cdot \gamma \cdot {{\left| \Updelta \right|} \mathord{\left/ {\vphantom {{\left| \Updelta \right|} {\phi^{ * } }}} \right. \kern-\nulldelimiterspace} {\phi^{ * } }}} \right) < 1.$ We can achieve this by requiring that $\gamma < {\frac{{p_{i}^{j} \left( 0 \right)}}{k \cdot \left| \Updelta \right|}} \cdot \phi^{ * } .$

Let

$$ \gamma = {\frac{{p_{i}^{j} \left( 0 \right)}}{2k \cdot \left| \Updelta \right|}} \cdot \phi^{ * } . $$

(18)

With this value of γ, Eq. (16) is simplified to Pr{Y ^j _i (k) ≤ M} < M · k ^m · ψ^k−m, where $\psi = 1 - \frac{1}{2}p_{i}^{j} \left( 0 \right),$ 0 < ψ < 1. Now we need to evaluate $\mathop { \lim }\limits_{k \to \infty } \;M \cdot k^{m} \cdot \psi^{k - m} .$

$\mathop { \lim }\limits_{k \to \infty } \;M \cdot k^{m} \cdot \psi^{k - m} = M \cdot \mathop { \lim }\limits_{k \to \infty } \;{\frac{{k^{m} }}{{\left( {{1 \mathord{\left/ {\vphantom {1 \psi }} \right. \kern-\nulldelimiterspace} \psi }} \right)^{k - m} }}},$ with $\gamma = {\frac{{p_{i}^{j} \left( 0 \right)}}{2k \cdot \left| \Updelta \right|}} \cdot \phi^{ * } .$

By applying l’Hopital’s rule:

$$ M \cdot \mathop { \lim }\limits_{k \to \infty } \;{\frac{{k^{m} }}{{\left( {{1 \mathord{\left/ {\vphantom {1 \psi }} \right. \kern-\nulldelimiterspace} \psi }} \right)^{k - m} }}} = M \cdot \mathop {\lim }\limits_{k \to \infty } \;{\frac{m!}{{\left( {\ln \left( {{1 \mathord{\left/ {\vphantom {1 \psi }} \right. \kern-\nulldelimiterspace} \psi }} \right)} \right)^{m} \left( {{1 \mathord{\left/ {\vphantom {1 \psi }} \right. \kern-\nulldelimiterspace} \psi }} \right)^{k - m} }}} = 0\quad {\text{with}}\;\gamma = {\frac{{p_{i}^{j} \left( 0 \right)}}{2k \cdot \left| \Updelta \right|}} \cdot \phi^{ * } $$

(19)

Therefore Eq. (16) has a limit of zero as $k \to \infty$ and γ → 0, whenever Eq. (18) is satisfied.

Since the limit exists, for every channel j there is a k(j) such that for all k > k(j), Eq. (16) holds.

Now set $\gamma \left( j \right) = {\frac{{p_{i}^{j} \left( 0 \right)}}{2k\left( j \right) \cdot \left| \Updelta \right|}} \cdot \phi^{ * } .$ It remains to be shown that Eq. (16) is satisfied for all γ < γ(j), and for all k > k(j). This is trivial because as γ decreases, the L.H.S. of Eq. (16) is monotonically decreasing, and so the inequality (16???) is preserved.

Also for any k > k(j), since $Y_{i}^{j} \left( {k\left( j \right)} \right) \ge M \Rightarrow Y_{i}^{j} \left( k \right) \ge M,$ by the laws of probability:

$$ { \Pr }\left\{ {Y_{i}^{j} \left( k \right) \ge M} \right\} \ge { \Pr }\left\{ {Y_{i}^{j} \left( {k\left( j \right)} \right) \ge M} \right\}. $$

(20)

Thus in this case also, the inequality (16???) still holds. Hence for any channel j, ${ \Pr }\left\{ {Y_{i}^{j} \left( k \right) \le M} \right\} \le \delta_{0}$ whenever k > k(j) and γ < γ(j). Since we can repeat this argument for all the channels, we can define k ₀ and γ ₀ as k ₀ = max_1≤j≤N {k(j)}, and $\gamma_{0} = \max_{1 \le j \le N} \left\{ {\gamma \left( j \right)} \right\}.$ Thus for all j, it is true that for all k > k ₀ and γ < γ ₀ (λ < λ ₀), the quantity ${ \Pr }\left\{ {Y_{i}^{j} \left( k \right) \le M} \right\} \le \delta_{0}$ and theorem is proved.□