Skip to main content
SpringerLink
Log in

Joint routing and channel assignment using online learning in cognitive radio networks

  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

Cognitive radio networks (CRNs) are the solution for the problem of underutilizing the licensed spectrum for which there are more requests in the last couple of decades. In CRNs, Secondary users (SUs) are permitted to access opportunistically the licensed spectrum owned by primary users (PUs). In this paper, we address the problem of joint routing and channel assignment for several flows generated by source SUs to a given destination. We consider a more realistic model based on Markov modulated Poisson process for modeling the PUs traffic at each channel and the SUs try to exploit short lived spectrum holes between the PUs packets at the selected channel. The SUs want to cooperatively minimize the end-to-end delay of source SUs flows meanwhile the quality of service requirements of the PUs would be met. To consider partial observation of SUs about PUs activity at all channels and quick adaptation of SUs decisions to environment changes and cooperative interaction of SUs, we use decentralized partially observable markov decision process for modeling the problem. Then, an online learning based scheme is proposed for solving the problem. Simulation results show that the performance of the proposed method and the optimal method is close to each other. Also, simulation results show that the proposed method greatly outperforms related works at control of interference to the PUs while maintains the end-to-end delay of SU flows in a low level.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Ying-Chang, L., Kwang-Cheng, C., Li, G. Y., & Mahonen, P. (2011). Cognitive radio networking and communications: An overview. IEEE Transactions on Vehicular Technology, 60, 3386–3407.

    Article  Google Scholar 

  2. Akyildiz, I. F., Lee, W.-Y., Vuran, M. C., & Mohanty, S. (2006). Next generation/dynamic spectrum access/cognitive radio wireless networks: A survey. Computer Networks, 50, 2127–2159.

    Article  MATH  Google Scholar 

  3. Wu, Y., Hu, F., Kumar, S., Matyjas, J. D., Sun, Q., & Zhu, Y. (2017). Apprenticeship learning based spectrum decision in multi-channel wireless mesh networks with multi-beam antennas. IEEE Transactions on Mobile Computing, 16(2), 314–325.

    Article  Google Scholar 

  4. Wellens, M., Riihijärvi, J., & Mähönen, P. (2009). Empirical time and frequency domain models of spectrum use. Physical Communication, 2(2), 10–32.

    Article  Google Scholar 

  5. How, K. C., Ma, M., & Qin, Y. (2011). Routing and QoS provisioning in cognitive radio networks. Computer Networks, 55, 330–342.

    Article  Google Scholar 

  6. Chowdhury, K. R., & Akyildiz, I. F. (2011). CRP: A routing protocol for cognitive radio ad hoc networks. IEEE Journal on Selected Areas in Communications, 29, 794–804.

    Article  Google Scholar 

  7. Ping, S., Aijaz, A., Holland, O., & Aghvami, A. H. (2015). SACRP: A spectrum aggregation-based cooperative routing protocol for cognitive radio ad-hoc networks. IEEE Transactions on Communications, 63, 2015–2030.

    Article  Google Scholar 

  8. Zhou, T. (2015). Joint routing and channel assignment for delay minimization in multi-channel multi-flow mobile cognitive ad hoc networks. In IEEE global communications conference (GLOBECOM) (pp. 1–6).

  9. Caleffi, M., Akyildiz, I. F., & Paura, L. (2012). OPERA: Optimal routing metric for cognitive radio ad-hoc networks. IEEE Transactions on Wireless Communications, 11, 2884–2894.

    Google Scholar 

  10. XiaoXia, H., Dianjie, L., Pan, L., & Yuguang, F. (2011). Coolest path: Spectrum mobility aware routing metrics in cognitive ad hoc networks. In International conference on distributed computing systems (ICDCS) (pp. 182–191).

  11. El-Sherif, A. A., & Mohamed, A. (2014). Joint routing and resource allocation for delay minimization in cognitive radio based mesh networks. IEEE Transactions on Wireless Communications, 13, 186–197.

    Article  Google Scholar 

  12. Filippini, I., Ekici, E., & Cesana, M. (2013). A new outlook on routing in cognitive radio networks: Minimum-maintenance-cost routing. IEEE/ACM Transactions on Networking, 21, 1484–1498.

    Article  Google Scholar 

  13. Tang, F., & Li, J. (2017). Joint rate adaptation, channel assignment and routing to maximize social welfare in multi-hop cognitive radio networks. IEEE Transactions on Wireless Communications, 16(4), 2097–2110.

    Article  Google Scholar 

  14. Bing, X., Wahab, M. H., Yang, Y., Zhong, F., & Sooriyabandara, M. (2009). Reinforcement learning based spectrum-aware routing in multi-hop cognitive radio networks. In International conference on cognitive radio oriented wireless networks and communications (CROWNCOM) (pp. 1–5).

  15. Saleem, Y., Yau, K. A., Mohamad, H., Ramli, N., & Rehmani, M. H. (2015). SMART: A spectrum-aware cluster-based routing scheme for distributed cognitive radio networks. Computer Networks, 91, 196–224.

    Article  Google Scholar 

  16. Wang, W., Kwasinski, A., & Niyato, D. (2016). Learning for robust routing based on stochastic game in cognitive radio networks. https://arxiv.org/abs/1603.08664.

  17. Quanyan, Z., Zhou, Y., Ju Bin, S., Zhu, H., & Basar, T. (2012). Interference aware routing game for cognitive radio multi-hop networks. IEEE Journal on Selected Areas in Communications, 30, 2006–2015.

    Article  Google Scholar 

  18. Heyman, D. P., & Lucantoni, D. (2003). Modeling multiple IP traffic streams with rate limits. IEEE/ACM Transactions on Networking, 11, 948–958.

    Article  Google Scholar 

  19. Fu, F., & van der Schaar, M. (2009) A systematic framework for dynamically optimizing multi-user video transmission. http://arxiv.org/abs/0903.0207.

  20. Fangwen, F., & van der Schaar, M. (2009). Learning to compete for resources in wireless stochastic games. IEEE Transactions on Vehicular Technology, 58, 1904–1919.

    Article  Google Scholar 

  21. Won, C. K., & Hossain, E. (2011). Opportunistic access to spectrum holes between packet bursts: A learning-based approach. IEEE Transactions on Wireless Communications, 10, 2497–2509.

    Article  Google Scholar 

  22. Won, C. K., & Hossain, E. (2013). Estimation of primary user parameters in cognitive radio systems via hidden Markov model. IEEE Transactions on Signal Processing, 61, 782–795.

    Article  MathSciNet  MATH  Google Scholar 

  23. Shiang, H.-P., & van der Schaar, M. (2010). Online learning in autonomic multi-hop wireless networks for transmitting mission-critical applications. IEEE Journal on Selected Areas in Communications, 28(5), 728–741.

    Article  Google Scholar 

  24. Yagan, D., & Chen-Khong, T. (2007). Coordinated reinforcement learning for decentralized optimal control. In IEEE international symposium on approximate dynamic programming and reinforcement learning (pp. 296–302).

  25. Perkins, C. E., & Bhagwat, P. (1994). Highly dynamic destination-sequenced distance-vector routing (DSDV) for mobile computers. SIGCOMM Computer Communication Review, 24, 234–244.

    Article  Google Scholar 

  26. Meuleau, N., Kim, K.-E., Kaelbling, L. P., & Cassandra, A. R. (1999). Solving POMDPs by searching the space of finite policies. In Presented at the proceedings of the fifteenth conference on uncertainty in artificial intelligence, Stockholm, Sweden.

  27. Koller, D., & Megiddo, N. (1992). The complexity of zero-sum games in extensive form. Games and Economic Behavior, 4(4), 528–552.

    Article  MathSciNet  MATH  Google Scholar 

  28. Bkassiny, M., Yang, L., & Jayaweera, S. K. (2013). A survey on machine-learning techniques in cognitive radios. Communications Surveys and Tutorials, IEEE, 15, 1136–1159.

    Article  Google Scholar 

  29. Aberdeen, D., & Baxter, J. (2002). Scaling internal-state policy-gradient methods for POMDPs. In Proceedings of the nineteenth international conference on machine learning (ICML’02), Morgan Kaufmann, Sydney, Australia.

  30. Baxter, J., & Bartlett, P. L. (2001). Infinite-horizon policy-gradient estimation. Journal on Artificial Intelligence Research, 15, 319–350.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Engineering Department, Amirkabir University of Technology, Tehran, Iran

    Babak Pourpeighambar, Mehdi Dehghan & Masoud Sabaei

Authors
  1. Babak Pourpeighambar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mehdi Dehghan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Masoud Sabaei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mehdi Dehghan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pourpeighambar, B., Dehghan, M. & Sabaei, M. Joint routing and channel assignment using online learning in cognitive radio networks. Wireless Netw 25, 2407–2421 (2019). https://doi.org/10.1007/s11276-018-1672-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11276-018-1672-9

Keywords

Search

Navigation