Abstract
This paper deals with the jamming attack which may hinder the cognitive radio from efficiently exploiting the spectrum. We model the problem of channel selection as a Markov decision process. We propose a real-time reinforcement learning algorithm based on Q-learning to pro-actively avoid jammed channels. The proposed algorithm is based on wideband spectrum sensing and a greedy policy to learn an efficient real-time strategy. The learning approach is enhanced through cooperation with the receiving CR node based on its sensing results. The algorithm is evaluated through simulations and real measurements with software defined radio equipment. Both simulations and radio measurements reveal that the presented solution achieves a higher packet success rate compared to the classical fixed channel selection and best channel selection without learning. Results are given for various scenarios and diverse jamming strategies.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Mitola III, J., & Jr Maguire, G. Q. (1999). Cognitive radio: Making software radios more personal. IEEE Personal Communications Magazine, 6, 13–18.
Haykin, S. (2005). Cognitive radio: Brain-empowered wireless communications. IEEE Journal on Selected Areas in Communications, 23, 201–220.
Akhtar, F., Rehmani, M. H., & Reisslein, M. (2016). White space: Definitional perspectives and their role in exploiting spectrum opportunities. Telecommunications Policy, 40, 319–331.
Saleem, Y., Salim, F., & Rehmani, M. H. (2015). Routing and channel selection from cognitive radio networks perspective: A survey. Computers & Electrical Engineering, 42, 117–134.
Wang, W., Bhattacharjee, S., Chatterjee, M., & Kwiat, K. (2013). Collaborative jamming and collaborative defense in cognitive radio networks. Pervasive and Mobile Computing, 9, 572–587.
Asterjadhi, A., & Zorzi, M. (2010). JENNA: A jamming evasive network-coding neighbor-discovery algorithm for cognitive radio networks. IEEE Wireless Communications, 17, 24–32.
Balogun, V. (2014). Anti-jamming performance of hybrid FEC code in the presence of CRN random jammers. International Journal of Novel Research in Engineering and Applied Sciences (IJNREAS), 1, 1.
Bhunia, S., Su, X., Sengupta, S., Vázquez-Abad, F. J. (2014). Stochastic model for cognitive radio networks under jamming attacks and honeypot-based prevention. In Distributed computing and networking—15th international conference (ICDCN ’14), Coimbatore, India (pp. 438-452), January 4–7, 2014
Wang, B., Wu, Y., Liu, K. J. R., & Charles, C. T. (2011). An anti-jamming stochastic game for cognitive radio networks. IEEE Journal on Selected Areas in Communications, 29, 877–889.
Dabcevic, K., Betancourt, A., Marcenaro, L., Regazzoni, C. S. (2014). A fictitious play-based game-theoretical approach to alleviating jamming attacks for cognitive radios. In IEEE international conference on acoustics, speech and signal processing (ICASSP)
Cadeau, W., Li, X., & Xiong, C. (2014). Markov model based jamming and anti-jamming performance analysis for cognitive radio networks. Communications and Network, 6, 76.
Amjad, M., Akhtar, F., Rehmani, M. H., Reisslein, M., & Umer, T. (2017). Full-duplex communication in cognitive radio networks: A survey. IEEE Communications Surveys and Tutorials, 19(4), 2158–2191.
Jia, L., Yao, F., Sun, Y., Xu, Y., Feng, S., & Anpalagan, A. (2017). A hierarchical learning solution for anti-jamming stackelberg game with discrete power strategies. IEEE Wireless Communication Letters, 6, 818–821.
Szepesvári, C., & Littman, M. L. (1996). Generalized markov decision processes: Dynamic-programming and reinforcement-learning algorithms. Providence, RI: Brown University.
Galindo-Serrano, A., & Giupponi, L. (2010). Distributed Q-learning for aggregated interference control in cognitive radio networks. IEEE Transactions on Vehicular Technology, 59, 1823–1834.
Wu, Y., Wang, B., & Liu, K. J. R. (2010). Optimal defense against jamming attacks in cognitive radio networks using the Markov decision process approach. In GLOBECOM’10 (pp. 1–5)
Chen, C., Song, M., Xin, C., & Backens, J. (2013). A game-theoretical anti-jamming scheme for cognitive radio networks. IEEE Network, 27, 22–27.
Slimeni, F., Scheers, B., Chtourou, Z., Le Nir, V. (2015). Jamming mitigation in cognitive radio networks using a modified Q-learning algorithm. In International conference on military communications and information systems (ICMCIS) (pp. 1–7), May 2015
Sutton, R. S., & Barto, A. G. (1998). Introduction to reinforcement learning. Cambridge, MA: MIT Press.
Watkins, C. J. C. H. (1989). Learning from delayed rewards. Cambridge: King’s College.
Abounadi, J., Bertsekas, D. P., & Borkar, V. (2002). Stochastic approximation for nonexpansive maps: Application to Q-learning algorithms. SIAM Journal on Control and Optimization, 41, 1–22.
Even-Dar, E., & Mansour, Y. (2004). Learning rates for Q-learning. Journal of Machine Learning Research, 5, 1–25.
Sun, H., Nallanathan, A., Wang, C. X., & Chen, Y. (2013). Wideband spectrum sensing for cognitive radio networks: A survey. IEEE Wireless Communications, 20, 74–81.
Le Nir, V., Scheers, B. (2015). Evaluation of open-source software frameworks for high fidelity simulation of cognitive radio networks. In International conference on military communications and information systems (ICMCIS) (pp. 1–6), May 2015
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Slimeni, F., Chtourou, Z., Scheers, B. et al. Cooperative Q-learning based channel selection for cognitive radio networks. Wireless Netw 25, 4161–4171 (2019). https://doi.org/10.1007/s11276-018-1737-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11276-018-1737-9