Mobile Edge Computing Against Smart Attacks with Deep Reinforcement Learning in Cognitive MIMO IoT Systems

Abstract

In wireless Internet of Things (IoT) systems, the multi-input multi-output (MIMO) and cognitive radio (CR) techniques are usually involved into the mobile edge computing (MEC) structure to improve the spectrum efficiency and transmission reliability. However, such a CR based MIMO IoT system will suffer from a variety of smart attacks from wireless environments, even the MEC servers in IoT systems are not secure enough and vulnerable to these attacks. In this paper, we investigate a secure communication problem in a cognitive MIMO IoT system comprising of a primary user (PU), a secondary user (SU), a smart attacker and several MEC servers. The target of our system design is to optimize utility of the SU, including its efficiency and security. The SU will choose an idle MEC server that is not occupied by the PU in the CR scenario, and allocates a proper offloading rate of its computation tasks to the server, by unloading such tasks with proper transmit power. In such a CR IoT system, the attacker will select one type of smart attacks. Then two deep reinforcement learning based resource allocation strategies are proposed to find an optimal policy of maximal utility without channel state information(CSI), one of which is the Dyna architecture and Prioritized sweeping based Edge Server Selection (DPESS) strategy, and the other is the Deep Q-network based Edge Server Selection (DESS) strategy. Specifically, the convergence speed of the DESS scheme is significantly improved due to the trained convolutional neural network (CNN) by utilizing the experience replay technique and stochastic gradient descent (SGD). In addition, the Nash equilibrium and existence conditions of the proposed two schemes are theoretically deduced for the modeled MEC game against smart attacks. Compared with the traditional Q-learning algorithm, the average utility and secrecy capacity of the SU can be improved by the proposed DPESS and DESS schemes. Numerical simulations are also presented to verify the better performance of our proposals in terms of efficiency and security, including the higher convergence speed of the DESS strategy.

Appendix : A: Proof of theorem 1

Proof

By (2), we have

$$ U(0, p)=xL- \rho e_{0}- \nu t_{0} -xL \frac{\rho p+ \nu}{B\log_{2} \det \left( \mathbf{I}+ \frac{p}{N_{U}} {\mathbf{H}}_{UM} {\mathbf{H}}_{UM}^{T} \right)}. $$

(10)

Let A and function f(p) denote xL − ρe₀ − νt₀ and $-xL \frac {\log _{2} \det \left (\mathbf {I}+ \frac {p}{N_{U}} {\mathbf {H}}_{UM} {\mathbf {H}}_{UM}^{T} \right )}{\rho p+ \nu }$, respectively. Thus, we can obtain the equivalent expression of the utility as

$$ U=A+1/(Bf(p)). $$

(11)

By the expression of f(p), it is clear that we can get

$$ \frac{\partial f(p)}{\partial p}|_{p=0} = - \frac{\text{Tr}[{\mathbf{H}}_{UM} {\mathbf{H}}_{UM}^{T}]}{N_{M} xL\nu b\ln 2}<0. $$

(12)

By (9), we can obtain

$$ \frac{\partial f(p)}{\partial p}|_{p=\overline{p}} <0. $$

(13)

If the equality (8) holds, we have $\frac {\partial ^{2} f(p)}{\partial p^{2}}>0$. Thus, $\frac {\partial f(p)}{\partial p}$ increases with p monotonically and it is always negative. From (11), we get

$$ \frac{\partial U}{\partial p}=(\frac{-\partial f(p)}{\partial p})/(Bf^{2}(p))>0. $$

(14)

Therefore, it can be shown that the utility of the SU increases with p. Moreover, we can have the maximum of U_U(0,p), i.e., $U(0, \overline {p})$. Therefore, from (14),

$$ \overline{p} = \arg\max \limits_{\underline{p}\leq p\leq \tilde{p}} U_{U}(0, p) $$

(15)

holds for $\overline {p}$.

By the definition of (1), we have

$$ U_{A}(0, \overline{p}) = -\log_{2} \det \left( \mathbf{I}+ \frac{\overline{p}}{N_{U}} {\mathbf{H}}_{UM} {\mathbf{H}}_{UM}^{T} \right) -\omega_{0}, $$

(16)

$$ U_{A}(1, \overline{p}) = -\log_{2} \det \left( \mathbf{I}+ \frac{\overline{p}}{N_{U}} {\mathbf{H}}_{UM} {\mathbf{H}}_{UM}^{T} \right) + \log_{2} \det \left( \mathbf{I}+ \frac{p_{S}}{N_{A}} {\mathbf{H}}_{MA} {\mathbf{H}}_{MA}^{T} \right) -\omega_{1}, $$

(17)

$$ U_{A}(2, \overline{p}) = - \log_{2} \det \left( \frac{\mathbf{I}+ \frac{\overline{p}}{N_{U}} {\mathbf{H}}_{UM} {\mathbf{H}}_{UM}^{T}}{\mathbf{I}+ \frac{p_{J}}{N_{A}} {\mathbf{H}}_{MA} {\mathbf{H}}_{MA}^{T}} \right)-\omega_{2}, $$

(18)

$$ \begin{array}{@{}rcl@{}} U_{A}(3, \overline{p}) = &&-\log_{2} \det \left( \mathbf{I}+ \frac{\overline{p}}{N_{U}} {\mathbf{H}}_{UM} {\mathbf{H}}_{UM}^{T} \right)\\ && +\log_{2} \det \left( \mathbf{I}+ \frac{\overline{p}}{N_{U}} {\mathbf{H}}_{UA} {\mathbf{H}}_{UA}^{T} \right)-\omega_{3}. \end{array} $$

(19)

If the equations (5)-(7) hold, we can get

$$ U_{A}(0, \overline{p})-U_{A}(1, \overline{p}) \geq 0, $$

(20)

$$ U_{A}(0,\overline{p})-U_{A}(2, \overline{p}) \geq 0, $$

(21)

$$ U_{A}(0, \overline{p})-U_{A}(3, \overline{p}) \geq 0. $$

(22)

Therefore, the equality

$$ 0 = \arg \max \limits_{g\in \mathcal{A}_{A}}U_{A}(g,\overline{p}) $$

(23)

holds for g^∗ = 0.

Then by (15) and (23), we can conclude that the NE of the MEC system is $(\overline {p},0)$. By (9), we obtain

$$ \begin{array}{@{}rcl@{}} \frac{\partial U_{U}}{\partial x}=1&&-\frac{\rho p +\nu}{B\log2 \det (\mathbf{I}+\frac{p}{N_{U}}\mathbf{H}_{UM}\mathbf{H}_{UM}^{T})} \\ &&+ \phi(\rho e_{0} +\nu t_{0}) > \phi (\rho kf^{2} +\nu f^{-1})>0, \end{array} $$

(24)

$$ \frac{\partial^{2} U_{U}}{\partial x^{2}}=0, $$

(25)

which implies that the utility of the SU increases with x when NE holds. Thus, we can know the utility can arrive at the maximum when x is maximal, i.e.,

$$ \begin{array}{@{}rcl@{}} U_{U}=\overline{x}L(1&&-\frac{\rho \overline{p}+\nu }{B\log_{2} {\det} (1+ \frac{ \overline{p}}{N_{U}}\mathbf{H}_{UM}\mathbf{H}_{UM}^{T})})\\ &&+L\phi (\rho kf^{2} +\nu f^{-1})(1-\overline{x}). \end{array} $$

(26)

Thus, the SU can get the optimal utility if the inequalities (5)-(9) hold. □