Skip to main content
SpringerLink
Log in

Reinforcement learning for cooperative sensing gain in cognitive radio ad hoc networks

  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

Spectrum sensing is a fundamental function in cognitive radio networks for detecting the presence of primary users in licensed bands. The detection performance may be considerably compromised due to multipath fading and shadowing. To resolve this issue, cooperative sensing is an effective approach to combat channel impairments by cooperation of secondary users. This approach, however, incurs overhead such as delay for reporting local decisions and the increase of control traffic. In this paper, a reinforcement learning-based cooperative sensing (RLCS) method is proposed to address the cooperation overhead problem and improve cooperative gain in cognitive radio ad hoc networks. The proposed algorithm is proven to converge and capable of (1) finding the optimal set of cooperating neighbors with minimum control traffic, (2) minimizings the overall cooperative sensing delay, (3) selecting independent users for cooperation under correlated shadowing, and (4) excluding unreliable users and data from cooperation. Simulation results show that the RLCS method reduces the overhead of cooperative sensing while effectively improving the detection performance to combat correlated shadowing. Moreover, it adapts to environmental change and maintains comparable performance under the impact of primary user activity, user movement, user reliability, and control channel fading.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Lo, B. F., & Akyildiz, I. F. (2010). Reinforcement learning-based cooperative sensing in cognitive radio ad hoc networks. In Proceedings of IEEE PIMRC, pp. 2244–2249.

  2. Akyildiz, I. F., Lee, W. Y., & Chowdhury, K. R. (2009). CRAHNs: Cognitive radio ad hoc networks. Ad Hoc Networks Journal (Elsevier), 7, 810.

    Article  Google Scholar 

  3. Akyildiz, I. F., Lee, W. Y., Vuran, M. C., & Mohanty, S. (2006). NeXt generation/dynamic spectrum access/cognitive radio wireless networks: A survey. Computer Networks Journal (Elsevier), 50, 2127.

    Article  MATH  Google Scholar 

  4. Akyildiz, I. F., Lo, B. F., & Balakrishnan, R. (2011). Cooperative spectrum sensing in cognitive radio networks: A survey. Physical Communication (Elseviar) Journal, 4(1), 40.

    Article  Google Scholar 

  5. Cabric, D., Mishra, S. M., & Brodersen, R. W. (2004). Implementation issues in spectrum sensing for cognitive radios. In Proceedings of 38th Asilomar Conference on Signals, Systems, and Computers, pp. 772–776.

  6. Ghasemi, A., & Sousa, E. S. (2005). Collaborative spectrum sensing for opportunistic access in fading environments. In Proceedings of IEEE DySPAN, pp. 131–136.

  7. Lo, B. F., Akyildiz, I. F., & Al-Dhelaan, A. M. (2010). Efficient recovery control channel design in cognitive radio ad hoc networks. IEEE Transactions on Vehicular Technology, 59(9), 4513.

    Article  Google Scholar 

  8. Lo, B. F. (2011). A survey on common control channel design for cognitive radio networks. Physical Communication (Elseviar) Journal, 4(1), 26.

    Article  Google Scholar 

  9. Chen, R., Park, J. M., & Bian, K. (2008). Robust distributed spectrum sensing in cognitive radio networks. In Proceedings of IEEE INFOCOM, pp. 1876 –1884.

  10. Varshney, P. K. (1997). Distributed detection and data fusion. New York: Springer.

    Book  Google Scholar 

  11. Unnikrishnan, J., & Veeravalli, V. V. (2008). Cooperative sensing for primary detection in cognitive radio. IEEE Journal of Selected Topics in Signal Processing, 2(1), 18.

    Article  Google Scholar 

  12. Selen, Y., Tullberg, H., & Kronander, J. (2008). Sensor selection for cooperative spectrum sensing. In Proceedings of IEEE DySPAN, pp. 1–11.

  13. Visotsky, E., Kuffner, S., & Peterson, R. (2005). On collaborative detection of TV transmissions in support of dynamic spectrum sharing. In Proceedings of IEEE DySPAN, pp. 338–345.

  14. Ma, J., Zhao, G., & Li, Y. (2008). Soft combination and detection for cooperative spectrum sensing in cognitive radio networks. IEEE Transactions on Wireless Communications, 7(11), 4502.

    Article  Google Scholar 

  15. Visser, F. E., Janssen, G. J., & Pawelczak, P. (2008). Multinode spectrum sensing based on energy detection for dynamic spectrum access. In Proceedings of IEEE VTC-Spring 2008, pp. 1394–1398.

  16. Zhou, X., Ma, J., Li, G., Kwon, Y., & Soong, A. (2010). Probability-based combination for cooperative spectrum sensing. IEEE Transactions on Communications, 58(2), 463.

    Article  Google Scholar 

  17. Sun, C., Zhang, W., & Letaief, K. (2007). Cooperative spectrum sensing for cognitive radios under bandwidth constraints. In Proceedings of IEEE WCNC 2007 (pp. 1–5).

  18. Zhang, X., Qiu, Z., & Mu, D. (2008). Asynchronous cooperative spectrum sensing in cognitive radio. In Proceedings of IEEE ICSP, pp. 2020–2023.

  19. Song, C., & Zhang, Q. (2008). Sliding-window algorithm for asynchronous cooperative sensing in wireless cognitive networks. In Proceedings of IEEE ICC, pp. 3432–3436.

  20. Sutton, R., & Barto, A. (1988). Reinforcement learning: An introduction. Cambridge, MA: The MIT Press.

    Google Scholar 

  21. Berthold, U., Fu, F., van der Schaar, M., & Jondral, F. K. (2008). Detection of spectral resources in cognitive radios using reinforcement learning. In Proceedings of IEEE DySPAN.

  22. Di Felice, M., Chowdhury, K. R., Meleis, W. & Bononi, L. (2010). To sense or to transmit: A learning-based spectrum management scheme for cognitive radiomesh networks. In Proceedings of IEEE Workshop on Wireless Mesh Networks (WIMESH), pp. 1–6.

  23. Vucevic, N., Akyildiz, I. F., & Perez-Romero, J. (2011). Dynamic cooperator selection in cognitive radio networks. Ad Hoc Networks (Elsevier) Journal (to appear).

  24. Oksanen, J., Lundén, J., & Koivunen, V. (2010). Reinforcement learning-based multiband sensing policy for cognitive radios. In Proceedings of 2nd Int’l Workshop on Cognitive Information Processing (CIP), pp. 316–321.

  25. Oksanen, J., Lundén, J., & Koivunen, V. (2010). Reinforcement learning method for energy efficient cooperative multiband spectrum sensing. In Proceedings of IEEE Int’l Workshop on Machine Learning for Signal Processing (MLSP), pp. 59–64.

  26. Lee, W. Y., & Akyildiz, I. F. (2008). Optimal spectrum sensing framework for cognitive radio networks. IEEE Transactions on Wireless Communications, 7(10), 3845.

    Article  Google Scholar 

  27. Gudmundson, M. (1991). Correlation model for shadow fading in mobile radio systems. Electronics Letters, 27(23), 2145.

    Article  Google Scholar 

  28. Wang, H. S., & Moayeri, N. (1995). Finite-state markov channel–A useful model for radio communication channels. IEEE Transactions on Vehicular Technology, 44(1), 163.

    Article  Google Scholar 

  29. Singh, S., Jaakkola, T., Littman, M. L., & Szepesvári, C. (2000). Convergence results for single-step on-policy reinforcement learning algorithms. Machine Learning, 38(3), 287.

    Article  MATH  Google Scholar 

  30. Watkins, C. J. C. H. & Dayan, P. (1992). Q-learning. Machine Learning, 8, 279.

    MATH  Google Scholar 

  31. Breiman, L. (1968). Probability. Boston, MA: Addison-Wesley.

    MATH  Google Scholar 

  32. Puterman, M. L. (1994). Markov decision processes: Discrete stochastic dynamic programming. New York, NY: Wiley.

    Book  MATH  Google Scholar 

  33. Shiryayev, A. N. (1978). Optimal stopping rules. New York, NY: Springer.

    MATH  Google Scholar 

  34. Poor, H. V., & Hadjiliadis, O. (2009). Quickest Detection. Cambridge, UK: Cambridge University Press.

    MATH  Google Scholar 

Download references

Acknowledgments

This work was supported by the U.S. National Science Foundation under Award ECCS-0900930.

Author information

Authors and Affiliations

  1. Broadband Wireless Networking Laboratory, School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, 30332, USA

    Brandon F. Lo & Ian F. Akyildiz

  2. Faculty of Computing and Information Technology, King Abdul-Aziz University, P.O. Box 80221, Jeddah, 21589, Saudi Arabia

    Ian F. Akyildiz

Authors
  1. Brandon F. Lo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ian F. Akyildiz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Brandon F. Lo.

Additional information

A preliminary version of this work was presented at IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), Istanbul, Turkey, September 2010 [1].

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lo, B.F., Akyildiz, I.F. Reinforcement learning for cooperative sensing gain in cognitive radio ad hoc networks. Wireless Netw 19, 1237–1250 (2013). https://doi.org/10.1007/s11276-012-0530-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11276-012-0530-4

Keywords

Search

Navigation