Abstract
Due to the openness of the wireless propagation environment, wireless networks are highly susceptible to malicious jamming, which significantly impacts their legitimate communication performance. This study investigates a reconfigurable intelligent surface (RIS) assisted anti-jamming communication system. Specifically, the objective is to enhance the system’s anti-jamming performance by optimizing the transmitting power of the base station and the passive beamforming of the RIS. Taking into account the dynamic and unpredictable nature of a smart jammer, the problem of joint optimization of transmitting power and RIS reflection coefficients is modeled as a Markov decision process (MDP). To tackle the complex and coupled decision problem, we propose a learning framework based on the double deep Q-network (DDQN) to improve the system achievable rate and energy efficiency. Unlike most power-domain jamming mitigation methods that require information on the jamming power, the proposed DDQN algorithm is better able to adapt to dynamic and unknown environments without relying on the prior information about jamming power. Finally, simulation results demonstrate that the proposed algorithm outperforms multi-armed bandit (MAB) and deep Q-network (DQN) schemes in terms of the anti-jamming performance and energy efficiency.
摘要
由于无线传播环境的开放性, 无线网络极易受到恶意干扰, 严重影响其合法通信性能. 研究了一种基于可重构智能表面的抗干扰通信系统, 通过优化基站的发射功率和可重构智能表面的被动波束成形来提高系统的抗干扰性能. 考虑到智能干扰机的动态和不可预测性, 将发射功率和可重构智能表面反射系数的联合优化问题建模为马尔可夫决策过程. 为解决复杂和耦合的决策问题, 提出一种基于双深度Q网络的学习框架, 提高系统的可达速率和能量效率. 与大多数功率域的抗干扰方法需要干扰功率信息不同, 提出的双深度Q网络算法更能适应动态和未知的干扰环境, 而不依赖于关于干扰功率的先验信息. 仿真结果表明, 所提算法在抗干扰性能和能量效率方面均优于多臂赌博机算法和深度Q网络算法.
This is a preview of subscription content, log in via an institution to check access.
Data availability
The data that support the findings of this study are available from the corresponding author upon reasonable request.
References
Arulkumaran K, Deisenroth MP, Brundage M, et al., 2017. Deep reinforcement learning: a brief survey. IEEE Signal Process Mag, 34(6):26–38. https://doi.org/10.1109/MSP.2017.2743240
Basar E, di Renzo M, de Rosny J, et al., 2019. Wireless communications through reconfigurable intelligent surfaces. IEEE Access, 7:116753–116773. https://doi.org/10.1109/ACCESS.2019.2935192
Björnson E, Özdogan Ö, Larsson EG, 2020. Intelligent reflecting surface versus decode-and-forward: how large surfaces are needed to beat relaying? IEEE Wirel Commun Lett, 9(2):244–248. https://doi.org/10.1109/LWC.2019.2950624
Chang GY, Wang SY, Liu YX, 2017. A jamming-resistant channel hopping scheme for cognitive radio networks. IEEE Trans Wirel Commun, 16(10):6712–6725. https://doi.org/10.1109/TWC.2017.2728659
di Renzo M, Zappone A, Debbah M, et al., 2020. Smart radio environments empowered by reconfigurable intelligent surfaces: how it works, state of research, and the road ahead. IEEE J Select Areas Commun, 38(11):2450–2525. https://doi.org/10.1109/JSAC.2020.3007211
Feng ZB, Ren GC, Chen J, et al., 2019. Power control in relay-assisted anti-jamming systems: a Bayesian three-layer Stackelberg game approach. IEEE Access, 7:14623–14636. https://doi.org/10.1109/ACCESS.2019.2893459
Feng ZB, Luo YJ, Chen XQ, et al., 2020. A MAB-based discrete power control approach in anti-jamming relay communication via three-layer Stackelberg game. Proc 6th Int Conf on Computer and Communications, p.267–272. https://doi.org/10.1109/ICCC51575.2020.9344934
Geng SQ, Li PK, Yin XZ, et al., 2022. The study on antijamming power control strategy based on Q-learning. Proc 7th Int Conf on Intelligent Computing and Signal Processing, p.182–185. https://doi.org/10.1109/ICSP54964.2022.9778818
Guo HY, Liang YC, Chen J, et al., 2020. Weighted sum-rate maximization for reconfigurable intelligent surface aided wireless networks. IEEE Trans Wirel Commun, 19(5):3064–3076. https://doi.org/10.1109/TWC.2020.2970061
He KM, Zhang XY, Ren SQ, et al., 2016. Deep residual learning for image recognition. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.770–778. https://doi.org/10.1109/CVPR.2016.90
Huang CW, Zappone A, Alexandropoulos GC, et al., 2019. Reconfigurable intelligent surfaces for energy efficiency in wireless communication. IEEE Trans Wirel Commun, 18(8):4157–4170. https://doi.org/10.1109/TWC.2019.2922609
Jian MN, Alexandropoulos GC, Basar E, et al., 2022. Reconfigurable intelligent surfaces for wireless communications: overview of hardware designs, channel models, and estimation techniques. Intell Converg Netw, 3(1):1–32. https://doi.org/10.23919/ICN.2022.0005
Khawaja W, Ozdemir O, Yapici Y, et al., 2020. Coverage enhancement for NLOS mmWave links using passive reflectors. IEEE Open J Commun Soc, 1:263–281. https://doi.org/10.1109/OJCOMS.2020.2969751
Li XC, Chen JN, Ling X, et al., 2023. Deep reinforcement learning-based anti-jamming algorithm using dual action network. IEEE Trans Wirel Commun, 22(7):4625–4637. https://doi.org/10.1109/TWC.2022.3227575
Luong NC, Hoang DT, Gong SM, et al., 2019. Applications of deep reinforcement learning in communications and networking: a survey. IEEE Commun Surv Tut, 21(4):3133–3174. https://doi.org/10.1109/COMST.2019.2916583
Lyu L, Shen Y, Zhang SC, 2022. The advance of reinforcement learning and deep reinforcement learning. Proc IEEE Int Conf on Electrical Engineering, Big Data and Algorithms, p.644–648. https://doi.org/10.1109/EEBDA53927.2022.9744760
Ma N, Xu K, Xia XC, et al., 2022. Reinforcement learning-based dynamic anti-jamming power control in UAV networks: an effective jamming signal strength based approach. IEEE Commun Lett, 26(10):2355–2359. https://doi.org/10.1109/LCOMM.2022.3193309
Mnih V, Kavukcuoglu K, Silver D, et al., 2015. Human-level control through deep reinforcement learning. Nature, 518(7540):529–533. https://doi.org/10.1038/nature14236
Ning WL, Huang XY, Yang K, et al., 2020. Reinforcement learning enabled cooperative spectrum sensing in cognitive radio networks. J Commun Netw, 22(1):12–22. https://doi.org/10.1109/JCN.2019.000052
Pirayesh H, Zeng HC, 2022. Jamming attacks and anti-jamming strategies in wireless networks: a comprehensive survey. IEEE Commun Surv Tut, 24(2):767–809. https://doi.org/10.1109/COMST.2022.3159185
Ramachandran P, Zoph B, Le QV, 2017. Searching for activation functions. https://arxiv.org/abs/1710.05941
Sharma H, Kumar N, Tekchandani R, 2023. Mitigating jamming attack in 5G heterogeneous networks: a federated deep reinforcement learning approach. IEEE Trans Veh Technol, 72(2):2439–2452. https://doi.org/10.1109/TVT.2022.3212966
Shen ZX, Xu K, Xia XC, 2021. 2D fingerprinting-based localization for mmWave cell-free massive MIMO systems. IEEE Commun Lett, 25(11):3556–3560. https://doi.org/10.1109/LCOMM.2021.3109645
Summers TA, Wilson SG, 1998. SNR mismatch and online estimation in turbo decoding. IEEE Trans Commun, 46(4):421–423. https://doi.org/10.1109/26.664291
Sun YF, An K, Luo JS, et al., 2021. Intelligent reflecting surface enhanced secure transmission against both jamming and eavesdropping attacks. IEEE Trans Veh Technol, 70(10):11017–11022. https://doi.org/10.1109/TVT.2021.3104580
Sutton RS, Barto AG, 2018. Reinforcement Learning: an Introduction. MIT Press, Cambridge, USA.
Takizawa K, Sasaki S, Zhou J, et al., 2002. Online SNR estimation for parallel combinatorial SS systems in Nakagami fading channels. Proc Global Telecommunications Conf, p.1239–1243. https://doi.org/10.1109/GLOCOM.2002.1188395
Tang X, Wang DW, Zhang RN, et al., 2021. Jamming mitigation via aerial reconfigurable intelligent surface: passive beamforming and deployment optimization. IEEE Trans Veh Technol, 70(6):6232–6237. https://doi.org/10.1109/TVT.2021.3077662
van Hasselt H, Guez A, Silver D, 2016. Deep reinforcement learning with double Q-learning. Proc 30th AAAI Conf on Artificial Intelligence, p.2094–2100.
Wang PL, Fang J, Yuan XJ, et al., 2020. Intelligent reflecting surface-assisted millimeter wave communications: joint active and passive precoding design. IEEE Trans Veh Technol, 69(12):14960–14973. https://doi.org/10.1109/TVT.2020.3031657
Wang W, Zhang W, 2021. Joint beam training and positioning for intelligent reflecting surfaces assisted millimeter wave communications. IEEE Trans Wirel Commun, 20(10):6282–6297. https://doi.org/10.1109/TWC.2021.3073140
Wang W, Zhang W, 2022a. Intelligent reflecting surface configurations for smart radio using deep reinforcement learning. IEEE J Select Areas Commun, 40(8):2335–2346. https://doi.org/10.1109/JSAC.2022.3180787
Wang W, Zhang W, 2022b. Jittering effects analysis and beam training design for UAV millimeter wave communications. IEEE Trans Wirel Commun, 21(5):3131–3146. https://doi.org/10.1109/TWC.2021.3118558
Wei L, Huang CW, Alexandropoulos GC, et al., 2021. Channel estimation for RIS-empowered multi-user MISO wireless communications. IEEE Trans Commun, 69(6):4144–4157. https://doi.org/10.1109/TCOMM.2021.3063236
Wu QQ, Zhang R, 2019. Intelligent reflecting surface enhanced wireless network via joint active and passive beamforming. IEEE Trans Wirel Commun, 18(11):5394–5409. https://doi.org/10.1109/TWC.2019.2936025
Wu QQ, Zhang R, 2020. Towards smart and reconfigurable environment: intelligent reflecting surface aided wireless network. IEEE Commun Mag, 58(1):106–112. https://doi.org/10.1109/MCOM.001.1900107
Xiao L, Hong SY, Xu SY, et al., 2022. IRS-aided energy-efficient secure WBAN transmission based on deep reinforcement learning. IEEE Trans Commun, 70(6):4162–4174. https://doi.org/10.1109/TCOMM.2022.3169813
Xiao ZC, Gao B, Liu SC, et al., 2018. Learning based power control for mmWave massive MIMO against jamming. Proc IEEE Global Communications Conf, p.1–6. https://doi.org/10.1109/GLOCOM.2018.8647173
Xu JD, Yuen C, Huang CW, et al., 2023. Reconfiguring wireless environments via intelligent surfaces for 6G: reflection, modulation, and security. Sci China Inf Sci, 66(3):130304. https://doi.org/10.1007/s11432-022-3626-5
Xu JW, Wang KH, Zhang X, et al., 2021. Anti-jamming strategy based on game theory in single-channel UAV communication network. Proc 6th Int Conf on Fog and Mobile Edge Computing, p.1–7. https://doi.org/10.1109/FMEC54266.2021.9732602
Yang HL, Xiong ZH, Zhao J, et al., 2020. Intelligent reflecting surface assisted anti-jamming communications based on reinforcement learning. Proc IEEE Global Communications Conf, p.1–6. https://doi.org/10.1109/GLOBECOM42002.2020.9322599
Yang HL, Xiong ZH, Zhao J, et al., 2021a. Deep reinforcement learning-based intelligent reflecting surface for secure wireless communications. IEEE Trans Wirel Commun, 20(1):375–388. https://doi.org/10.1109/TWC.2020.3024860
Yang HL, Xiong ZH, Zhao J, et al., 2021b. Intelligent reflecting surface assisted anti-jamming communications: a fast reinforcement learning approach. IEEE Trans Wirel Commun, 20(3):1963–1974. https://doi.org/10.1109/TWC.2020.3037767
Yu L, Li YS, Pan C, et al., 2017. Anti-jamming power control game for data packets transmission. Proc 17th Int Conf on Communication Technology, p.1255–1259. https://doi.org/10.1109/ICCT.2017.8359836}
Zhang SW, Zhang R, 2020. Capacity characterization for intelligent reflecting surface aided MIMO communication. IEEE J Select Areas Commun, 38(8):1823–1838. https://doi.org/10.1109/JSAC.2020.3000814
Zhang ZD, Zhang DX, Qiu RC, 2020. Deep reinforcement learning for power system applications: an overview. CSEE J Power Energy Syst, 6(1):213–225. https://doi.org/10.17775/CSEEJPES.2019.00920
Author information
Authors and Affiliations
Contributions
Yang LIU designed the research, processed the data, and drafted the paper. Kui XU, Nan MA, and Jianhui XU helped organize the paper. Kui XU, Xiaochen XIA, and Wei XIE revised and finalized the paper.
Corresponding author
Ethics declarations
Yang LIU, Kui XU, Xiaochen XIA, Wei XIE, Nan MA, and Jianhui XU declare that they have no conflict of interest.
Additional information
Project supported by the Natural Science Foundation of Jiangsu Province, China (Nos. BK 20201334, BK 20200579, and BK 20231485), the National Natural Science Foundation of China (Nos. 62071485, 62271503, and 62001513), and the Basic Research Project of Jiangsu Province, China (No. BK 20192002)
Rights and permissions
About this article
Cite this article
Liu, Y., Xu, K., Xia, X. et al. Joint power control and passive beamforming optimization in RIS-assisted anti-jamming communication. Front Inform Technol Electron Eng 24, 1791–1802 (2023). https://doi.org/10.1631/FITEE.2200646
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/FITEE.2200646