Resource Allocation for Multi-source Multi-relay Wireless Networks: A Multi-Armed Bandit Approach

Abstract

In this paper, we consider the problem of link adaptation (rate allocation) of Orthogonal Multiple Access Multiple Relay Channel (OMAMRC) using the Multi-Armed Bandit (MAB) online learning framework. The cooperative system is composed of a transmission phase where sources transmit in a round robin manner, and a retransmission phase where a scheduled node sends redundancies. We assume that we have no knowledge of the Channel State Information (CSI) nor of the Channel Distributed Information (CDI). Accordingly, rate allocation must be learned online following a sequential learning algorithm. We adapt to one variant of the MAB framework algorithms, the Upper Confidence Bound (UCB) family, and specifically the UCB1 algorithm. The UCB1 algorithm achieves a logarithmic regret uniformly over time, without any preliminary knowledge about the reward distributions. Due to the exponential growth of the number of arms, following the multiple sources included in the rate allocation, the UCB1 algorithm features a complexity problem. Thus, we propose a sequential UCB1 (SUCB1) algorithm which solves the complexity issue, and outperforms the UCB1 algorithm.

Appendix

1.1 A: Outage Events

Based on [23] proposition 1, we see a direct relation between the individual outage and the common outage. The individual outage is defined as the event that an individual source is not decoded correctly at the destination after $T_{max}$ rounds. Similarly, common outage is defined for a set of sources, and it is declared when at least one of the sources within this set is not decoded correctly at the destination. In other words, common outage of a set occurs when one or more of its source nodes are in an individual outage.

Fig. 3.

Efficiency for $\gamma = -4$ dB

Fig. 4.

Efficiency for $\gamma = 6$ dB

Fig. 5.

Efficiency for $\gamma = 21$ dB

Fig. 6.

ASE vs $\gamma$ after 500 samples

Both, the individual outage event $\mathcal {O}_{s,t}(a_t,\mathcal {S}_{a_t,t-1} |\mathcal {P}_{t-1})$ of a source s after round t, and the common outage event $\mathcal {E}_t(a_t,\mathcal {S}_{a_t,t-1} |\mathcal {P}_{t-1})$ after round t, depend directly on the rate being scheduled. In addition, they depend on the selected node $a_t\in \mathcal {N}$ and its associated decoding set $\mathcal {S}_{a_t,t-1}$. They are conditional on the knowledge of $\mathcal {P}_{t-1}$, where $\mathcal {P}_{t-1}$ denotes the set collecting the nodes $\widehat{a}_l$ which were selected in rounds $l \in \{ 1,\dots ,t-1 \}$ prior to round t together with their associated decoding sets $\mathcal {S}_{\widehat{a}_l,l-1}$, and the decoding set of the destination $\mathcal {S}_{d,t-1}$ ($\mathcal {S}_{d,0}$ is the destination’s decoding set after the first phase).

Analytically, the common outage event of a given subset of sources is declared if the vector of their rates lies outside of the corresponding MAC capacity region. For some subset of sources $\mathcal {B}\subseteq \overline{\mathcal {S}}_{d,t-1}$, where $\overline{\mathcal {S}}_{d,t-1}=\mathcal {S} \setminus {\mathcal {S}}_{d,t-1}$ is the set of non-successfully decoded sources at the destination after round $t-1$, and for a candidate node $a_t$ this event can be expressed as:

$$\begin{aligned} \begin{aligned}&\mathcal {E}_{t,\mathcal {B}}(a_t,\mathcal {S}_{a_t,t-1})\\&=\bigcup _{\mathcal {U}\subseteq \mathcal {B}} \Big \{ \sum _{i \in \mathcal {U}}R_i > \sum _{i \in \mathcal {U}} I_{i,d} + \alpha \sum _{l=1}^{t-1} I_{\widehat{a}_l,d} \mathcal {C}_{\widehat{a}_l}(\mathcal {U}) + \alpha I_{a_t,d} \mathcal {C}_{a_t}(\mathcal {U}) \Big \}, \end{aligned} \end{aligned}$$

(4)

where $I_{a,b}$ denotes the mutual information between the nodes a and b (the mutual information is defined based on the channel inputs, check Sect. 6 for Gaussian inputs example), and where $\mathcal {C}_{\widehat{a}_l}$ and $\mathcal {C}_{a_t}$ have the following definitions:

$$\begin{aligned} \begin{aligned}&\mathcal {C}_{\widehat{a}_l}(\mathcal {U})=\Big [ (\mathcal {S}_{\widehat{a}_l,l-1}\cap \mathcal {U} \ne \emptyset )\wedge (\mathcal {S}_{\widehat{a}_l,l-1}\cap \mathcal {I}=\emptyset ) \Big ],\\&\mathcal {C}_{a_t}(\mathcal {U})=\Big [ (\mathcal {S}_{a_t,t-1}\cap \mathcal {U} \ne \emptyset )\wedge (\mathcal {S}_{a_t,t-1}\cap \mathcal {I}=\emptyset ) \Big ]. \end{aligned} \end{aligned}$$

(5)

The individual outage event of a source s after round t for a candidate node $a_t$ can be defined as:

$$\begin{aligned} \begin{aligned}&\mathcal {O}_{s,t}(a_t,\mathcal {S}_{a_t,t-1})=\bigcap _{\mathcal {I}\subset \overline{\mathcal {S}}_{d,t-1},\mathcal {B}= \overline{\mathcal {I}},s\in \mathcal {B}}\mathcal {E}_{t,\mathcal {B}}(a_t,\mathcal {S}_{a_t,t-1}), \\&=\bigcap _{\mathcal {I}\subset \overline{\mathcal {S}}_{d,t-1}} \bigcup _{\mathcal {U}\subseteq \overline{\mathcal {I}}:s\in \mathcal {U}} \Big \{ \sum _{i \in \mathcal {U}}R_i > \sum _{i \in \mathcal {U}} I_{i,d} + \alpha \sum _{l=1}^{t-1} I_{\widehat{a}_l,d} \mathcal {C}_{\widehat{a}_l}(\mathcal {U}) + \alpha I_{a_t,d} \mathcal {C}_{a_t}(\mathcal {U}) \Big \}, \end{aligned} \end{aligned}$$

(6)

where $\overline{\mathcal {I}}=\overline{\mathcal {S}}_{d,t-1}\setminus \mathcal {I}$.