Abstract
In this paper, the dynamic spectrum access (DSA) technique for an uplink underlay cognitive radio (CR) network is considered. The objective of the DSA scheme is to allow the secondary users (SUs) access the network on the premise of ensuring the quality of service of the primary user (PU). This DSA process is formulated as an optimization problem to maximize the sum rate of the SUs subject to the constraints of signal-to-interference-and-noise ratio (SINR) of both the PU and SUs, through adjusting the transmit powers and thus SINR thresholds of the SUs. Under the assumption of discrete feasible set, the formulated DSA problem is nonconvex and thus difficult to solve. We develop an intelligent solving method for this DSA problem based on Q-Learning. Numerical simulations show that the proposed algorithm can efficiently learn a solution that guarantees the link quality of the PU after allowing access of the SUs.
This work was supported in part by the National Key R&D Program of China under Grant 2019YFB2102600, the National Natural Science Foundation of China (NSFC) under Grants 61672321, 61701269, 61832012 and 61771289, the Key Research and Development Program of Shandong Province under Grants 2019JZZY020124 and 2019JZZY010313, and the Natural Science Foundation of Shandong Province under Grant ZR2017BF012.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Sun, S., Rappaport, T.S., Shafi, M., Tang, P., Zhang, J., Smith, P.J.: Propagation models and performance evaluation for 5G millimeter-wave bands. IEEE Trans. Veh. Technol. 67(9), 8422–8439 (2018)
Lu, J., Cai, Z., Wang, X., Zhang, L., Li, P., He, Z.: User social activity-based routing for cognitive radio networks. Pers. Ubiquit. Comput. 22(3), 471–487 (2018). https://doi.org/10.1007/s00779-018-1114-9
Wang, W., et al.: Joint precoding optimization for secure SWIPT in UAV-aided NOMA networks. IEEE Trans. Commun. 68, 5028–5040 (2020)
Clancy, C., Hecker, J., Stuntebeck, E., O’Shea, T.: Applications of machine learning to cognitive radio networks. IEEE Wirel. Commun. 14(4), 47–52 (2007)
Cai, Z., Ji, S., He, J., Wei, L., Bourgeois, A.G.: Distributed and asynchronous data collection in cognitive radio networks with fairness consideration. IEEE Trans. Parallel Distrib. Syst. 25(8), 2020–2029 (2014)
Bkassiny, M., Li, Y., Jayaweera, S.K.: A survey on machine-learning techniques in cognitive radios. IEEE Commun. Surv. Tutor. 15(3), 1136–1159 (2012)
Cai, Z., Ji, S., He, J., Bourgeois, A.G.: Optimal distributed data collection for asynchronous cognitive radio networks. In: IEEE International Conference on Distributed Computing Systems (2012)
Cai, Z., Duan, Y., Bourgeois, A.G.: Delay efficient opportunistic routing in asynchronous multi-channel cognitive radio networks. J. Comb. Optim. 29(4), 815–835 (2013). https://doi.org/10.1007/s10878-013-9623-y
Zhou, X., Sun, M., Li, G.Y., Juang, B.-H.F.: Intelligent wireless communications enabled by cognitive radio and machine learning. China Commun. 15(12), 16–48 (2018)
Aprem, A., Murthy, C.R., Mehta, N.B.: Transmit power control policies for energy harvesting sensors with retransmissions. IEEE J. Sel. Topics Signal Process. 7(5), 895–906 (2013)
Reddy, Y.B., Detecting primary signals for efficient utilization of spectrum using Q-learning. In: Fifth International Conference on Information Technology: New Generations (ITNG 2008), pp. 360–365. IEEE (2008)
Venkatraman, P., Hamdaoui, B., Guizani, M.: Opportunistic bandwidth sharing through reinforcement learning. IEEE Trans. Veh. Technol. 59(6), 3148–3153 (2010)
Galindo-Serrano, A., Giupponi, L.: Distributed Q-learning for aggregated interference control in cognitive radio networks. IEEE Trans. Veh. Technol. 59(4), 1823–1834 (2010)
Mohammadi F.S., Kwasinski, A.: QoE-driven integrated heterogeneous traffic resource allocation based on cooperative learning for 5G cognitive radio networks. In: IEEE 5G World Forum (5GWF), pp. 244–249. IEEE (2018)
Shah-Mohammadi, F., Kwasinski, A.: Deep reinforcement learning approach to QoE-driven resource allocation for spectrum underlay in cognitive radio networks. In: IEEE International Conference on Communications Workshops (ICC Workshops), pp. 1–6. IEEE (2018)
Kwasinski, A., Wang, W., Mohammadi, F.S.: Reinforcement learning for resource allocation in cognitive radio networks. In: Machine Learning for Future Wireless Communications, pp. 27–44 (2020)
Pietrzyk, S., Janssen, G.J.M.: Radio resource allocation for cellular networks based on OFDMA with QoS guarantees. In: IEEE Global Telecommunications Conference GLOBECOM 2004, vol. 4, pp. 2694–2699. IEEE (2004)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. The MIT Press, Cambridge (2018)
Watkins, C.J.C.H., Dayan, P.: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992)
Sutton, R.S.: Learning to predict by the methods of temporal differences. Mach. Learn. 3(1), 9–44 (1988)
Mendonca, M.R.F., Bernardino, H.S., Neto, R.F.: Reinforcement learning with optimized reward function for stealth applications. Entertainment Comput. 25, 37–47 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, J., Dong, A., Yu, J. (2020). Intelligent Dynamic Spectrum Access for Uplink Underlay Cognitive Radio Networks Based on Q-Learning. In: Yu, D., Dressler, F., Yu, J. (eds) Wireless Algorithms, Systems, and Applications. WASA 2020. Lecture Notes in Computer Science(), vol 12384. Springer, Cham. https://doi.org/10.1007/978-3-030-59016-1_57
Download citation
DOI: https://doi.org/10.1007/978-3-030-59016-1_57
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59015-4
Online ISBN: 978-3-030-59016-1
eBook Packages: Computer ScienceComputer Science (R0)