Abstract
In this paper, we propose a stochastic game that allows multiple secondary users for Quality of Service (QoS) support in cognitive radio networks. A state-transition model is built for the QoS provisioning transmission in these networks. At each stage of the game, secondary users observe the spectrum availability, channel quality and the strategy of the QoS provisioning transmission for all players. According to this observation, they will then decide how many channels they should be imposed to take into the QoS constraints. By using Q-learning, each group of secondary users can learn the optimal policy that maximises the expected sum of discounted payoff sum, defined as the spectrum-efficient throughput. The proposed stationary policy is shown to achieve much better performance than the policy obtained by ordinary stochastic games, which only maximise each stage’s payoff. Our results suggest that the proposed method can be used to obtain performance gains through better adaptation to the QoS limitations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Kang, X., Liang, Y.-C., Nallanathan, A.: Optimal Power Allocation for Fading Channels in Cognitive Radio Networks Under Transmit and Interference Power Constraints. In: IEEE Int. Conf. on Comm., pp. 3568–3572 (May 2008)
Attar, A., Nakhai, M.R., Aghvami, A.H.: Cognitive Radio Game: A Framework for Efficiency, Fairness and QoS Guarantee. In: IEEE Int. Conf. on Comm., pp. 4170–4174 (May 2008)
Newman, T.R., Barker, B.A., Wyglinski, A.M., Agah, A., Evans, J.B., Minden, G.J.: Cognitive Engine Implementation for Wireless Multicarrier Transceivers. Wiley Journal on Wireless Communications and Mobile Computing 7(9), 1129–1142 (2007)
Tsagkaris, K., Katidiots, A., Demestichas, P.: Neural Network-based Learning Schemes for Cognitive Radio Systems. Computer Communications 31, 3394–3404 (2008)
Kaniezhil, R., Kumar, C.D.N., Prakash, A.: Fuzzy Logic System for Opportunistic Spectrum Access using Cognitive Radio. UCSI International Journal of Computer Science 1(1), 703–709 (2013)
Tembine, H., Altman, E., El-Azouzi, R., Hayel, Y.: Evolutionary Games in Wireless Networks. IEEE Trans. on Systems, Man, and Cybernetics, Part B. Special Issues on Game Theory (2009)
Huang, J.-W., Krishnamurthy, V.: Dynamical Transmission Control. In: Zhang, Y., Guizani, M. (eds.) Game Theory for Wireless Communications and Networking. CRC Press, Boca Raton (2011)
Akin, S., Gursoy, M.C.: Effective Capacity of Cognitive Radio Channels for Quality of Service Provisioning. IEEE Trans. on Wireless Communications 9(11), 3354–3364 (2010)
Wu, D., Negi, R.: Effective Capacity: A Wireless A Wireless Link Model for Support of Quality of Service. IEEE Trans. on Wireless Communications 2(4), 630–643 (2003)
Watkins, C.J.C.H., Dayan, P.: Q-learning. Machine Learning 8, 279–292 (1992)
Littman, M.L.: Markov Games as a Framework for Multi-Agent Reinforcement Learning. In: Proc. 11th International Conference on Machine Learning, pp. 157–163 (1994)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Martyna, J. (2014). Q-Learning Algorithm Used by Secondary Users for QoS Support in Cognitive Radio Networks. In: Ali, M., Pan, JS., Chen, SM., Horng, MF. (eds) Modern Advances in Applied Intelligence. IEA/AIE 2014. Lecture Notes in Computer Science(), vol 8481. Springer, Cham. https://doi.org/10.1007/978-3-319-07455-9_41
Download citation
DOI: https://doi.org/10.1007/978-3-319-07455-9_41
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-07454-2
Online ISBN: 978-3-319-07455-9
eBook Packages: Computer ScienceComputer Science (R0)