Skip to main content

Advertisement

Springer Nature Link
Log in

Opportunistic channel access with repetition time diversity and switching cost: a block multi-armed bandit approach

  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

In this paper, we investigate the channel access problem considering switching cost in the block fading channels with unknown information of channel occupation and quality. We formulate this problem as a multi-armed bandit (MAB) problem with the goal of minimizing the outage rate and avoiding frequent channel switching. To achieve this goal, a block based multi-armed bandit (BMAB) learning algorithm is proposed. Furthermore, the BMAB algorithm is extended to cope with the short-term deep channel fading, by exploiting the repetition time diversity (RTD). The regrets of the proposed two algorithms are proved to be logarithmic in time. Performance analysis and simulation results show that the proposed algorithms outperform standard SMAB algorithm in average system outage rate, switching cost and throughput. In addition, the repetition time diversity multi-armed bandit (RTDMAB) algorithm is better than BMAB algorithm in the presence of deep channel fading at the cost of receiving complexity .

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Mitola, J., & Maguire, G. Q. (1999). Cognitive radio: Making software radios more personal. IEEE Personal Communications, 6(4), 13–18.

    Article  Google Scholar 

  2. Haykin, S. (2005). Cognitive radio: Brain-empowered wireless communications. IEEE Journal on Selected Areas in Communications, 23(2), 201–220.

    Article  Google Scholar 

  3. Liang, Y. C., Zeng, Y., Peh, E. C. Y., & Hoang, A. T. (2008). Sensing throughput tradeoff for cognitive radio networks. IEEE Transactions on Wireless Communications, 7(4), 1326–1337.

    Article  Google Scholar 

  4. Jiang, H., Lai, L., Fan, R., & Poor, H. V. (2009). Optimal selection of channel sensing order in cognitive radio. IEEE Transactions on Wireless Communications, 8(1), 297–307.

    Article  Google Scholar 

  5. Quan, Z., Cui, S., Sayed, A. H., & Poor, H. V. (2009). Optimal multiband joint detection for spectrum sensing in cognitive radio networks. IEEE Transactions on Signal Processing, 57(3), 1128–1140.

    Article  MathSciNet  Google Scholar 

  6. Fan, R., & Jiang, H. (2009). Channel sensing-order setting in cognitive radio networks: A two-user case. IEEE Transactions on Vehicular Technology, 58(9), 4997–5008.

    Article  Google Scholar 

  7. Zhang, R., Cui, S., & Liang, Y.-C. (2009). On ergodic sum capacity of fading cognitive multiple-access and broadcast channels. IEEE Transactions on Information Theory, 55(11), 5161–5178.

    Article  MathSciNet  MATH  Google Scholar 

  8. Fan, R., & Jiang, H. (2010). Optimal multi-channel cooperative sensing in cognitive radio networks. IEEE Transactions on Wireless Communications, 9(3), 1128–1138.

    Article  Google Scholar 

  9. Fan, R., Jiang, H., Guo, Q., & Zhang, Z. (2011). Joint optimal cooperative sensing and resource allocation in multichannel cognitive radio networks. IEEE Transactions on Vehicular Technology, 60(2), 722–729.

    Article  Google Scholar 

  10. Lai, L., El Gamal, H., Jiang, H., & Poor, H. V. (2011). Cognitive medium access: Exploration, exploitation, and competition. IEEE Transactions on Mobile Computing, 10(2), 239–253.

    Article  Google Scholar 

  11. Gao, S., Qian, L., Vaman, D. R., & Han, Z. (2010). Distributed cognitive sensing for time varying channels: Exploration and exploitation. In Proceeding on IEEE wireless communications and networking conference.

  12. Fang, X., Yang, D., & Xue, G. (2013). Taming wheel of fortune in the air: An algorithmic framework for channel selection strategy in cognitive radio networks. IEEE Transactions on Vehicular Technology, 62(2), 783–796.

    Article  Google Scholar 

  13. Anandkumar, A., Michael, N., Tang, A. K., & Swami, A. (2011). Distributed algorithms for learning and cognitive medium access with logarithmic regret. IEEE Journal on Selected Areas in Communications, 29(4), 731–745.

    Article  Google Scholar 

  14. Auer, P., Cesa-Bianchi, N., & Fischer, P. (2002). Finite-time analysis of the multiarmed bandit problem. Machine Learning, 47, 235–256.

    Article  MATH  Google Scholar 

  15. Cao, B., et al. (2014). Cooperative media access control with optimal relay selection in error-prone wireless networks. IEEE Transactions on Vehicular Technology, 63(1), 252–265.

    Article  Google Scholar 

  16. Cao, B., et al. (2015). Dynamic cooperative media access control for wireless networks. Wireless Communications and Mobile Computing, 15(13), 1759–1772.

    Article  Google Scholar 

  17. Li, Y., et al. (2011). A distributed cooperative MAC for cognitive radio ad-hoc networks. In IEEE symposium on computers and communications (ISCC).

  18. Li, Y., et al. (2015). Cooperative spectrum sharing with energy-save in cognitive radio networks. In IEEE global communications conference (GLOBECOM).

  19. Li, B., Yang, P., Wang, J., et al. (2012). Optimal action point for dynamic spectrum utilization under Rayleigh fading. Ad Hoc & Sensor Wireless Networks, 17(1–2), 1–32.

    Google Scholar 

  20. Sabharwal, A., Khoshnevis, A., & Knightly, E. (2007). Opportunistic spectral usage: Bounds and a multi-band CSMA/CA protocol. IEEE/ACM Transactions on Networking, 15(3), 533–545.

    Article  Google Scholar 

  21. Du, Z., Wu, Q., & Yang, P. (2014). Learning with handoff cost constraint for network selection in heterogeneous wireless network. Wireless Communications and Mobile Computing. doi:10.1002/wcm.2525.

    Google Scholar 

  22. Xu, Y., Anpalagan, A., Wu, Q., Shen, L., Gao, Z., & Wang, J. (2013). Decision-theoretic distributed channel selection for opportunistic spectrum access: Strategies, challenges and solutions. IEEE Communication Survey & Tutorials, 15(4), 1689–1713.

    Article  Google Scholar 

  23. Huang, J., Gan, X., & Feng, X. (2013). Multi-armed bandit based opportunistic channel access: A consideration of switch cost. In Proceeding on IEEE ICC.

  24. Agrawal, R., Teneketzis, D., & Anantharam, V. (1988). Asymptotically efficient adaptive allocation rules for the multiarmed bandit problem with switching. IEEE Transactions on Automatic Control, 33(10), 899–906.

    Article  MathSciNet  MATH  Google Scholar 

  25. Chen, L., Iellamo, S., & Coupechoux, M. (2011). Opportunistic spectrum access with channel switching cost for cognitive radio networks, In Proceeding on IEEE ICC.

  26. Jun, T. (2004). A survey on the bandit problem with switching costs. De Economist, 152(4), 513–541.

    Article  Google Scholar 

  27. Qin, Z., et al. (2015). Multiple armed bandits based opportunistic channel access with switching cost in block fading channel. In 2015 international conference IEEE wireless communications & signal processing (WCSP).

  28. Chen, D., Yin, S., Zhang, Q., Liu, M., & Li, S. (2009). Mining spectrum usage data: A large-scale spectrum measurement study. In ACM MOBICOM (p. 13C24).

  29. Caire, G., & Tuninetti, D. (2001). The throughput of hybrid-ARQ protocols for the Gaussian collision channel. IEEE Transactions on Information Theory, 47(5), 1971–1988.

    Article  MathSciNet  MATH  Google Scholar 

  30. Wang, J., Young Park, S., Senior, D. J. L., & Zoltowski, M. D. (2010). Throughput delay tradeoff for wireless multicast using hybrid-ARQ protocols. IEEE Transactions on Communications, 58(9), 2741–2751.

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported in part by the National Natural Science Foundation of China under Grant Nos. 61401508 and 61601490.

Author information

Authors and Affiliations

  1. China National Digital Switching System Engineering & Technological R&D Center, Zhengzhou, 450002, China

    Zhiqiang Qin, Youming Sun & Yuhua Xu

  2. College of Communications Engineering, PLA University of Science and Technology, Nanjing, 210007, China

    Jinlong Wang & Jin Chen

  3. PLA Academy of National Defence Information, Wuhan, 430010, China

    Zhiyong Du

Authors
  1. Zhiqiang Qin
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jinlong Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Jin Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Youming Sun
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Zhiyong Du
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Yuhua Xu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Yuhua Xu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qin, Z., Wang, J., Chen, J. et al. Opportunistic channel access with repetition time diversity and switching cost: a block multi-armed bandit approach. Wireless Netw 24, 1683–1697 (2018). https://doi.org/10.1007/s11276-016-1428-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11276-016-1428-3

Keywords