Skip to main content

Advertisement

SpringerLink
Log in

Reinforcement Learning Based Full-Duplex Cognitive Anti-jamming Using Improved Energy Detector

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

In this paper, reinforcement learning (RL) based cognitive anti-jamming system employing full duplex tactical radio is investigated under electromagnetic spectrum warfare scenario. Firstly, the analytical expressions of jamming sensing based on improved energy detection are derived to calculate the reward metric. Then, we propose the multidomain anti-jamming strategies based on different learning algorithm and the accurate reward. Simulation results indicate that learning-based cognitive anti-jamming strategies may increase about 25% of the throughput of tactical radio. Moreover, the upper confidence bound and Thompson sampling strategies almost have the same performance and they are superior to other RL anti-jamming schemes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Clark, B., & Gunzinger, M. (2015). Winning the air waves—Regaining America’s dominance in the electromagnetic spectrum. Center for Strategic and Budgetary Assessments (CSBA), US.

  2. Wang, B., Wu, Y., Liu, K. R., & Clancy, T. C. (2011). An anti-jamming stochastic game for cognitive radio networks. IEEE Journal on Selected Areas in Communications,29(4), 877–889.

    Article  Google Scholar 

  3. Choe, S., Ko, E., & Jeon, S. (2015). Tactical data link simulator with cognitive anti-jamming capability. In International conference on modeling, simulation and visualization methods (pp. 72–77).

  4. Singh, S., & Trivedi, A. (2012). Anti-jamming in cognitive radio networks using reinforcement learning algorithms. In 2012 ninth international conference on wireless and optical communications networks (WOCN), Indore, India (pp. 1–5).

  5. Bhattarai, S., Jerry Park, J.-M., Gao, B., Bian, K., & Lehr, W. (2016). An overview of dynamic spectrum sharing: Ongoing initiatives, challenges, and a roadmap for future research. IEEE Transactions on Cognitive Communications and Networking,2(2), 110–128.

    Article  Google Scholar 

  6. Aref, M. A., Jayaweera, S. K., & Machuzak, S. (2017, March). Multi-agent reinforcement learning based cognitive anti-jamming. In IEEE wireless communications and networking conference (WCNC’17), San Francisco, CA.

  7. Liu, X., Xu, Y., Jia, L., Wu, Q., et al. Anti-jamming communications using spectrum waterfall: A deep reinforcement learning approach. IEEE Communications Letters, 22(5), 998–1001.

    Article  Google Scholar 

  8. Xiao, L., Li, Y., Liu, J., & Zhao, Y. (2015). Power control with reinforcement learning in cooperative cognitive radio networks against jamming. Journal of Supercomputing,71, 3237–3257.

    Article  Google Scholar 

  9. Han, G., Xiao, L, & Poor, H.V. (2017). Two-dimensional anti-jamming communication based on deep reinforcement learning. In IEEE international conference on acoustics (pp. 2087–2091).

  10. Amjad, M., Akhtar, F., & Rehmani, M. H. (2017). Full-duplex communication in cognitive radio networks: A survey. IEEE Communications Surveys & Tutorials,19(4), 2158–2191.

    Article  Google Scholar 

  11. Bkassiny, M., Jayaweera, S. K., Li, Y., & Avery, K. A. (2012). Wideband spectrum sensing and non-parametric signal classification for autonomous self-learning cognitive radios. IEEE Transactions on Wireless Communications,11(7), 2596–2605.

    Article  Google Scholar 

  12. Wanuga, K., Gulati, N., Saarnisaari, H., & Dandekar, K. R. (2014, June). Online learning for spectrum sensing and reconfigurable antenna control. In 2014 9th international conference on cognitive radio oriented wireless networks and communications (CROWNCOM) (pp. 508–513).

  13. Garnaev, A., Liu, Y., & Trappe, W. (2016). Anti-jamming strategy versus a low-power jamming attack when intelligence of adversary’s attack type is unknown. IEEE Transactions on Signal and Information Processing over Networks,2(1), 49–56.

    Article  MathSciNet  Google Scholar 

  14. Wang, Y., Xu, Y., Shen, L., Xu, C., & Cheng, Y. (2014). Two-dimensional POMDP-based opportunistic spectrum access in time-varying environment with fading channels. Journal of Communications and Networks,16(2), 217–226.

    Article  Google Scholar 

  15. Li, H., & Qian, Y. (2017). Effects of IQ imbalance for simultaneous transmit and receive based cognitive anti-jamming receiver. AEU - International Journal of Electronics and Communications,72, 26–32.

    Article  Google Scholar 

  16. Riihonen, T., Korpi, D., Rantula, O., & Valkama, M. (2017, May). On the prospects of full-duplex military radios. In International conference on military communications and information systems, Oulu, Finland.

  17. Li, J., & H. Li. (2016). On the performance of cognitive anti-jamming communication system based on improved energy. In 2nd international conference on communication and information processing, ICCIP (pp. 222–226).

  18. Li, H., & Qian, Y. (2017). Transmit–sense–receive-based cognitive antijamming for airborne ad hoc networks. International Journal of Communication Systems,30(16), 1–10.

    Article  Google Scholar 

  19. Bonnefoi, R., Besson, L., Moy, C., Kaufmann, E., & Palicot, J. (2017). Multi-armed bandit learning in IoT networks and non-stationary settings. In 12th EAI conference on cognitive radio oriented wireless networks (CrownCom), Portugal (pp. 173–185).

  20. Wilhelmi, F., Cano, C., Neu, G., Bellalta, B., Jonsson, A., & Barrachina-Muñoz, S. (2019). Collaborative spatial reuse in wireless networks via selfish multi-armed bandits. Ad Hoc Networks,88(15), 129–141.

    Article  Google Scholar 

  21. Wilhelmi Roca, F., Bellalta, B., Cano Bastidas, C., & Jonsson, A. (2017). Implications of decentralized Q-learning resource allocation in wireless networks. In IEEE 28th annual international symposium on personal, indoor, and mobile radio communications (PIMRC), Montreal, Canada (pp. 8–13).

Download references

Funding

Funding was provided by Program of the Aeronautical Science Foundation of China under Grant 2018ZC15003.

Author information

Authors and Affiliations

  1. Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China

    Haitao Li, Jiawei Luo & Jiayu Li

Authors
  1. Haitao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiawei Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiayu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haitao Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, H., Luo, J. & Li, J. Reinforcement Learning Based Full-Duplex Cognitive Anti-jamming Using Improved Energy Detector. Wireless Pers Commun 111, 2107–2127 (2020). https://doi.org/10.1007/s11277-019-06974-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-019-06974-y

Keywords

Search

Navigation