On cognitive processes in cognitive radio networks

Abstract

In this article we model the cognitive processes and evaluate their impact on the performance of cognitive radio networks (CRN). Operation of the cognitive radio nodes, can be characterized by two types of processes: communication processes such as packets transmission, and cognitive processes such as estimation of the network state and decision-making for dynamic resource allocation. We propose a continuous time Markov chain model of CRN that couples these processes into unified queueing framework and analyze it by means of the matrix-geometric approach. From the obtained results, we derive the performance measures of CRN such as average delay and throughput, and establish their dependencies on the underlying cognitive processes. Additionally, we design an efficient policy for accessing the vacant channels and managing the transmission-sensing trade-off, which arises when transmissions and sensing are mutually exclusive. The policy search is carried out by the stochastic optimization method of cross-entropy. The optimized policy leads to significantly enhanced performance of CRN.

Appendices

Appendix 1: Analysis of the 3-D CTMC

We present here the analysis of the 3-D CTMC in Fig. 4.

1.1 CTMC structure

In order to make the analysis of the system easier we numerate the states of Z _t lexicographically, i.e. (0, 0), (0, 1), …, (0, M), (1, 0), (1, 1), …, (M, M) and index them 1 to (M + 1)². This new order of states turns our CTMC to two dimensional since now Z _t  ∈ {1, 2, …, (M + 1)²}. Then again we order the states lexicographically, i.e. (0, 1), (0, 2), …, (0, M + 1), (1, 1), (1, 2), … and construct the generator matrix Q of this CTMC which is given by:

where B ₀₀ = {B ₀₀(i, j)}, B ₀₁ = {B ₀₁(i, j)}, B ₁₀ = {B ₁₀(i, j)}, B ₁₁ = {B ₁₁(i, j)}, A ₀ = {A ₀(i, j)}, A ₁ = {A ₁(i, j)} and A ₂ = {A ₂(i, j)} are (M + 1)² × (M + 1)² matrices. A 0 entry in Q (and in other matrices) is a matrix of all zeros of the appropriate dimension. It can be seen that in our model B ₀₁ = A ₀ = diag{λ, λ, …, λ}. For each value z _i,j  = (i, j) the process Z _t can take, the service rate is μ _i,j  = μ min{i, j}. We order the elements μ _i,j in the same way as we did for Z _t and obtain a vector of service rates μ. It can be seen that B ₁₀ = A ₂ = diag{μ}, while the matrices B ₀₀ and B ₁₁ = A ₁ are more complicated:

$$ B_{00} \left( {i,j} \right) = \left\{ {\begin{array}{*{20}c} { - \left( {\lambda + \left( {M - \left\lfloor {i/\left( {M + 1} \right)} \right\rfloor } \right)\alpha + \left\lceil {i/\left( {M + 1} \right)} \right\rceil \beta } \right)} \hfill & {\quad j = i} \hfill \\ {(M - \left\lfloor {i/\left( {M + 1} \right)} \right\rfloor )\alpha } \hfill & {\quad j = i + M} \hfill \\ {\left\lceil {i/\left( {M + 1} \right)} \right\rceil \beta } \hfill & {\quad j = i - M} \hfill \\ \delta \hfill & {\quad j = \left\lfloor {i/\left( {M + 1} \right)} \right\rfloor \left( {M + 2} \right) \cap i \ne j} \hfill \\ 0 \hfill & {\quad {\text{else}}} \hfill \\ \end{array} } \right. $$

and

$$ A_{1} (i,j) = \left\{ {\begin{array}{*{20}c} { - \left( {\lambda + i\mu + \left( {M - \left\lfloor {i/\left( {M + 1} \right)} \right\rfloor } \right)\alpha + \left\lceil {i/\left( {M + 1} \right)} \right\rceil \beta } \right)} \hfill & {\quad j = i} \hfill \\ {\left( {M - \left\lfloor {i/\left( {M + 1} \right)} \right\rfloor } \right)\alpha } \hfill & {\quad j = i + M} \hfill \\ {\left\lceil {i/\left( {M + 1} \right)} \right\rceil \beta } \hfill & {\quad j = i - M} \hfill \\ \delta \hfill & {\quad j = \left\lfloor {i/\left( {M + 1} \right)} \right\rfloor \left( {M + 2} \right) \cap i \ne j} \hfill \\ 0 \hfill & {\quad {\text{else}}} \hfill \\ \end{array} } \right. $$

1.2 Stationary probabilities

We define the stationary probabilities π _i,j of the process to be at level i and state j within that level. Calculating the stationary probabilities will allow evaluating interesting quantities, mainly the average delay of SU. The calculations here follow [23] and are adopted for our model.

Let π _i  ≡ (π _i,1, π _i,2, …, π _i,M² ) and π ≡ (π ₀, π ₁, π ₂, …). The stationary distribution is the unique set of π _i  ≥ 0, i ≥ 0, that solves

$$ \left\{ {\begin{array}{*{20}c} {\pi Q = \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{0} } \\ {\pi \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{e} \,\, = \,\,1} \\ \end{array} } \right. $$

(10)

where e (0) denotes an appropriately dimensioned column (row) vector of 1’s (0’s). From the first equation in (10) we may write down for the repeating portion of the process (j ≥ 1):

$$ \pi_{j - 1} A_{0} + \pi_{j} A_{1} + \pi_{j + 1} A_{2} = \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{0} \quad (j \ge 1) $$

(11)

For this type of CTMC characterized by a boundary conditions in the first column of Q followed by a repetitive portion of columns containing matrices A ₀, A ₁ and A ₂, there exist some constant matrix R such that

$$ \pi_{j} = \pi_{j - 1} R,\quad \quad (j \ge 1) $$

(12)

and that the values of π _j , j ≥ 1, have a matrix geometric form:

$$ \pi_{j} = \pi_{0} R^{j} ,\quad (j \ge 1) $$

(13)

substituting (13) into (11) yields

$$ A_{0} + RA_{1} + R^{2} A_{2} = 0 $$

(14)

This quadratic equation in R is typically solved numerically. There is more than one R that solves (14). When the CTMC is ergodic, there is an unique stationary distribution π that satisfies (10). Analogous to the scalar case where the utilization factor should be less than unity, in our case all eigenvalues of R must be less then unity for the normalization constraint in (10) to hold [24].

After solving for R, in order to determine the stationary probabilities, we continue with the boundary conditions:

$$ \pi_{0} B_{00} + \pi_{1} A_{2} = \pi_{0} (B_{00} + RA_{2} ) = 0 $$

(15)

Equation (15) alone is not enough to solve for π ₀ since it is not of full rank and we must use the normalization constraint in (10):

$$ \pi e\,\, = \,\,\left( {\sum\limits_{j = 0}^{\infty } {\pi_{j} } } \right)e = \pi_{0} (I - R)^{ - 1} e = 1 $$

(16)

Combining (15) and (16) we have

$$ \pi_{0} \left[ {\left( {I - R} \right)^{ - 1} \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{e} ,\,\left( {B_{00} + RA_{2} } \right)^{*} } \right] = \left[ {1,\underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{0} } \right] $$

(17)

where $\left( {B_{00} + RA_{2} } \right)^{*}$ is the result from removal of the first column from the matrix (B ₀₀ + RA ₂), and [1,0] is a row vector consisting of a 1 followed by (M + 1)² − 1 zeros. Equation (17) is solved by appropriate numerical methods.

Appendix 2: Cross-entropy algorithm for CRN policy optimization

In this appendix we present the CE algorithm for CRN policy optimization.

Input:

• function W(P; Ω)

• system parameters Ω = {α, β, M, λ, μ}

• probability density families {p _C (·; σ _C )} and {p _θ (·; σ _θ )},

• initial parameters σ _C,0 and σ _θ,0

• parameters N, J, T, d, ε

• t ← 0

Repeat

• 1: Generate samples C ^(j) (j = 1, 2, …, J) from p _C (·; σ _C,t-1)

• 2: Generate samples θ ^(j) (k = 1, 2, …, J) from p _θ (·; σ _θ,t-1)

• 3: Compose policy samples P ^(j) = (C ^(j),θ ^(j)) (j = 1, 2, …, J)

• 4: Calculate W ^(j) = W(P ^(j); Ω) for each sample (j = 1, 2, …, J)

• 5: Keep N (N < J) best samples graded by their W ^(j) value and discard the other samples

• 6: V _t  = min _j (W ^(j)) (minimize over the saved N best samples)

• 7: Using the N best samples update the parameters

  • 7.1: $\sigma_{C,t} \leftarrow \arg \mathop {\hbox{max} }\limits_{{\sigma_{C} }} \sum\limits_{n = 1}^{N} {\ln \left( {p_{{\sigma_{C} }} \left( {C^{(n)} ;\sigma_{C} } \right)} \right)}$

  • 7.2: $\sigma_{\theta ,t} \leftarrow \arg \mathop {\hbox{max} }\limits_{{\sigma_{\theta } }} \sum\limits_{n = 1}^{N} {\ln \left( {p_{{\sigma_{\theta } }} \left( {\theta^{(n)} ;\sigma_{\theta } } \right)} \right)}$

• 8: t ← t + 1

Until (t > T or |V _t  − V _t-τ |<ε, τ = 1, 2, …, d)

Output:

P ^* = (C ^*, θ ^*)—best sample, W ^* = W(P ^*; Ω)—best value