Abstract
With the continuous development of network communication and the application of many specific scenarios, the dynamics of network traffic continues to increase, making the optimization of routing problems an NP-hard problem. When using traditional routing algorithms, accuracy and efficiency cannot be balanced. Recently, Deep Q-Network (DQN) has shown great potential for solving dynamic network problems. However, existing DQN-based routing solutions often overlook network environment issues related to packet level, packet size, expected transmission time, and do not generalize well when the network changes. In this paper, we present a new carrier sense multiple access (CSMA) protocol called MC-DQN CSMA, which employs Deep Q-Network to improve the performance of the network. First, we propose a distance constraint under the signal-to-interference-to-noise ratio (SINR) model, which effectively avoids interference and improves the probability of success. Based on the dynamic and unpredictable needs of Ad Hoc networks, we try to use DQN strategy to train the network’s agents without expert knowledge. Furthermore, we demonstrate the performance of the proposed algorithm by comparing it with other methods and describing it graphically, which focus on transmitting packets in multi-channel Ad hoc networks.
Supported by Jiangxi Province 03 Special Project (No. 20203ABC03W07).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Wang, Y.-L., Mei, S., Wei, Y.-F., Wang, Y.-H., Wang, X.-J.: Improved ant colony-based multi-constrained QoS energy-saving routing and throughput optimization in wireless Ad-hoc networks. J. Chin. Univ. Posts Telecommun. 21, 43–59 (2014)
Luo, J., Zhang, J., Yu, L., Wang, X.: Impact of location popularity on throughput and delay in mobile Ad hoc networks. IEEE Trans. Mobile Comput. 14, 1004–1017 (2015)
Mahdian, M., Yeh, E.M.: Throughput and delay scaling of content-centric Ad hoc and heterogeneous wireless networks. IEEE/ACM Trans. Netw. 25, 3030–3043 (2017)
Zafar, S., Tariq, H., Manzoor, K.: Throughput and delay analysis of AODV, DSDV and DSR routing protocols in mobile Ad hoc networks. Int. J. Comput. Netw. Appl. (IJCNA), 3, 1–7 (2016)
Huang, K.: Spatial throughput of mobile Ad hoc networks powered by energy harvesting. IEEE Trans. Inf. Theory 59, 7597–7612 (2013)
Wang, C., Yu, J., Yu, D., Huang, B., Yu, S.: An improved approximation algorithm for the shortest link scheduling in wireless networks under SINR and hypergraph models. J. Combinatorial Optimiz. 32, 1052–1067 (2014)
Huang, B., Yu, J., Cheng, X., Chen, H., Liu, H.: SINR based shortest link scheduling with oblivious power control in wireless networks. J. Netw. Comput. Appl. 77, 64–72 (2017)
Yu, J., Huang, B., Cheng, X., Atiquzzaman, M.: Shortest link scheduling algorithms in wireless networks under the SINR model. IEEE Trans. Vehicular Technol. 66, 2643–2657 (2016)
Yu, D., Wang, Y., Hua, Q., Yu, J., Lau, F.: Distributed wireless link scheduling in the SINR model. J. Combinat. Optimiz. 32, 278–292 (2016). https://doi.org/10.1007/s10878-015-9876-8
Yu, D., Ning, L., Zou, Y., Yu, J., Cheng, X., Lau, F.C.: Distributed spanner construction with physical interference: constant stretch and linear sparseness. IEEE/ACM Trans. Netw. 25, 2138–2151 (2017)
Zhang, X., Yu, J., Li, W., Cheng, X., Yu, D., Zhao, F.: Localized algorithms for Yao graph-based spanner construction in wireless networks under SINR. IEEE/ACM Trans. Netw. 25, 2459–2472 (2017)
Yu, J., Jia, L., Li, W., Cheng, X., Wang, S., Bie, R., Yu, D.: A self-stabilizing algorithm for CDS construction with constant approximation in wireless networks under SINR model. 2015 IEEE 35th International Conference on Distributed Computing Systems, pp. 792–793. IEEE (2015)
Tian, X., Yu, J., Ma, L., Li, G., Cheng, X.: Distributed deterministic broadcasting algorithms under the SINR model. In: IEEE INFOCOM 2016-The 35th Annual IEEE International Conference on Computer Communications, pp. 1–9. IEEE (2016)
Ning, L., Yu, D., Zhang, Y., Wang, Y., Lau, F., Feng, S.: Uniform information exchange in multi-channel wireless Ad hoc networks. arXiv preprint arXiv:1503.08570 (2015)
Daum, S., Ghaffari, M., Gilbert, S., Kuhn, F., Newport, C.: Maximal independent sets in multichannel radio networks. In: Proceedings of the 2013 ACM Symposium on Principles of Distributed Computing, pp. 335–344 (2013)
Shi, W., Hua, Q.-S., Yu, D., Wang, Y., Lau, F.C.M.: Efficient information exchange in single-hop multi-channel radio networks. In: Wang, X., Zheng, R., Jing, T., Xing, K. (eds.) WASA 2012. LNCS, vol. 7405, pp. 438–449. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31869-6_38
Wang, Y., Wang, Y., Yu, D., Yu, J., Lau, F.: Information exchange with collision detection on multiple channels. J. Combinat. Optimiz. 31, 118–135 (2016). https://doi.org/10.1007/s10878-014-9713-5
Wu, S.-L., Lin, C.-Y., Tseng, Y.-C., Sheu, J.-L.: A new multi-channel mac protocol with on-demand channel assignment for multi-hop mobile Ad hoc networks. Proceedings International Symposium on Parallel Architectures, Algorithms and Networks. I-SPAN 2000, pp. 232–237. IEEE (2000)
Inokuchi, M., Ueda, H., Motoyoshi, G.: Throughput improvement by disruption-suppressed channel switching in multi-channel Ad-hoc networks. In: 2017 International Conference on Computing, Networking and Communications (ICNC), pp. 615–619. IEEE (2017)
Hu, T., Fei, Y.: QELAR: a machine-learning based adaptive routing protocol for energy-efficient and lifetime-extended underwater sensor networks. IEEE Trans. Mob. Comput. 9, 796–809 (2010)
Sun, P., Li, J., Guo, Z., Xu, Y., Lan, J., Hu, Y.: SINET: enabling scalable network routing with deep reinforcement learning on partial nodes. In: Proceedings of the ACM SIGCOMM 2019 Conference Posters and Demos, pp. 88–89 (2019)
Sun, P., Hu, Y., Lan, J., Tian, L., Chen, M.: TIDE: time-relevant deep reinforcement learning for routing optimization. Future Gener. Comput. Syst. 99, 401–409 (2019)
Xu, Q., Zhang, Y., Wu, K., Wang, J., Lu, K.: Evaluating and boosting reinforcement learning for intra-domain routing. In: 2019 IEEE 16th International Conference on Mobile Ad Hoc and Sensor Systems (MASS), pp. 265–273. IEEE (2019)
Acknowledgements
This work was supported by Jiangxi artificial intelligence production-education integration innovation center.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Pi, X., Qiu, J. (2024). Multi-channel Deep Q-network Carrier Sense Multiple Access. In: Vaidya, J., Gabbouj, M., Li, J. (eds) Artificial Intelligence Security and Privacy. AIS&P 2023. Lecture Notes in Computer Science, vol 14509. Springer, Singapore. https://doi.org/10.1007/978-981-99-9785-5_6
Download citation
DOI: https://doi.org/10.1007/978-981-99-9785-5_6
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-9784-8
Online ISBN: 978-981-99-9785-5
eBook Packages: Computer ScienceComputer Science (R0)