Abstract
With the reduction in manufacturing and launch costs of low Earth orbit satellites and the advantages of large coverage and high data transmission rates, satellites have become an important part of data transmission in air-ground networks. However, due to the factors such as geographical location and people’s living habits, the differences in user’ demand for multimedia data will result in unbalanced network traffic, which may lead to network congestion and affect data transmission. In addition, in traditional satellite network transmission, the convergence of network information acquisition is slow and global network information cannot be collected in a fine-grained manner, which is not conducive to calculating optimal routes. The service quality requirements cannot be satisfied when multiple service requests are made. Based on the above, in this paper artificial intelligence technology is applied to the satellite network, and a software-defined network is used to obtain the global network information, perceive network traffic, develop comprehensive decisions online through reinforcement learning, and update the optimal routing strategy in real time. Simulation results show that the proposed reinforcement learning algorithm has good convergence performance and strong generalizability. Compared with traditional routing, the throughput is 8% higher, and the proposed method has load balancing characteristics.
摘要
随着低轨卫星制造和发射成本的降低, 以及其覆盖范围大、 数据传输速率高等优点, 低轨卫星已成为空地网络数据传输的重要组成部分. 但受地理位置及人们生活习惯等因素影响, 用户对数据需求差异会造成网络流量不均衡, 可能导致网络拥塞进而影响数据传输. 传统卫星网络获取网络信息收敛慢, 无法细粒度收集全局网络信息, 不利于计算最优路由. 多业务请求无法满足服务质量要求. 本文将人工智能技术应用于低轨卫星网络, 利用软件定义网络获取全局网络信息, 感知网络流量, 通过强化学习在线制定综合决策, 实时更新最优路由策略. 仿真结果表明, 所提强化学习算法有良好收敛性和较强泛化能力. 与传统路由相比, 本文算法吞吐量提高了8%, 且具有负载均衡性.
This is a preview of subscription content, log in via an institution to check access.
Data availability
The data that support the findings of this study are available from the corresponding author upon reasonable request.
References
Arfeen A, Uddin R, 2020. Quality of experience-based optimization of satellite Internet-at-sea using WAN accelerators. Int J Satell Commun Netw, 38(6):527–556. https://doi.org/10.1002/sat.1366
Bujari A, Luglio M, Palazzi CE, et al., 2020. A virtual PEP for web optimization over a satellite-terrestrial backhaul. IEEE Commun Mag, 58(10):42–48. https://doi.org/10.1109/MCOM.001.2000322
Chen Q, Yang L, Guo DK, et al., 2022. LEO satellite networks: when do all shortest distance paths belong to minimum hop path set. IEEE Trans Aerosp Electron Syst, 58(4):3730–3734. https://doi.org/10.1109/TAES.2022.3143090
Gao K, Xu CQ, Qin JR, et al., 2019. QoS-driven path selection for MPTCP: a scalable SDN-assisted approach. Proc IEEE Wireless Communications and Networking Conf, p.1-6. https://doi.org/10.1109/WCNC.2019.8885585
Han C, Huo LY, Tong XH, et al., 2020. Spatial anti-jamming scheme for Internet of satellites based on the deep reinforcement learning and Stackelberg game. IEEE Trans Veh Technol, 69(5):5331–5342. https://doi.org/10.1109/TVT.2020.2982672
Huo LW, Jiang DD, Zhu XN, et al., 2022. A SDN-based fine-grained measurement and modeling approach to vehicular communication network traffic. Int J Commun Syst, 35(12):e4092. https://doi.org/10.1002/dac.4092
Jia ZY, Sheng M, Li JD, et al., 2020. LEO-satellite-assisted UAV: joint trajectory and data collection for Internet of Remote Things in 6G aerial access networks. IEEE Int Things J, 8(12):9814–9826. https://doi.org/10.1109/JIOT.2020.3021255
Kuhn N, Michel F, Thomas L, et al., 2020. QUIC: opportunities and threats in SATCOM. Proc 10th Advanced Satellite Multimedia Systems Conf and the 16th Signal Processing for Space Communications Workshop, p.1-7.
Langley A, Riddoch A, Wilk A, et al., 2017. The QUIC transport protocol: design and Internet-scale deployment. Proc Conf of the ACM Special Interest Group on Data Communication, p.183-196. https://doi.org/10.1145/3098822.3098842
Li X, Tang FL, Zhu YM, et al., 2022. Processing-while-transmitting: cost-minimized transmission in SDN-based STINs. IEEE/ACM Trans Netw, 30(1):243–256. https://doi.org/10.1109/TNET.2021.3107413
Liu D, Zhang JK, Cui JJ, et al., 2022. Deep learning aided routing for space-air-ground integrated networks relying on real satellite, flight, and shipping data. IEEE Wirel Commun, 29(2):177–184. https://doi.org/10.1109/MWC.003.2100393
Liu JH, Zhao BK, Xin Q, et al., 2021. DRL-ER: an intelligent energy-aware routing protocol with guaranteed delay bounds in satellite mega-constellations. IEEE Trans Netw Sci Eng, 8(4):2872–2884. https://doi.org/10.1109/TNSE.2020.3039499
Liu LT, Shen YL, Zeng SG, et al., 2021. FO-Sketch: a fast oblivious sketch for secure network measurement service in the cloud. Electronics, 10(16):2020. https://doi.org/10.3390/electronics10162020
Liu ZG, Zhu J, Zhang JM, et al., 2020. Routing algorithm design of satellite network architecture based on SDN and ICN. Int J Satell Commun Netw, 38(1):1–15. https://doi.org/10.1002/sat.1304
Mogensen RS, Markmoller C, Madsen TK, et al., 2019. Selective redundant MP-QUIC for 5G mission critical wireless applications. Proc IEEE 89th Vehicular Technology Conf, p.1-5. https://doi.org/10.1109/VTCSpring.2019.8746482
Murua J, Reviriego P, 2020. Faking elephant flows on the count min sketch. IEEE Netw Lett, 2(4):199–202. https://doi.org/10.1109/LNET.2020.3035272
Oroojlooyjadid A, Nazari M, Snyder LV, et al., 2022. A deep Q-network for the beer game: deep reinforcement learning for inventory optimization. Manuf Ser Oper Manag, 24(1):285–304. https://doi.org/10.1287/msom.2020.0939
Rabitsch A, Hurtig P, Brunstrom A, 2018. A stream-aware multipath QUIC scheduler for heterogeneous paths. Proc Workshop on the Evolution, Performance, and Interoperability of QUIC, p.29-35. https://doi.org/10.1145/3284850.3284855
Shi H, Zhang L, Zuo XT, et al., 2021. Multipath deadline-aware transport proxy for space network. IEEE Int Comput, 25(6):51–57. https://doi.org/10.1109/MIC.2021.3112804
Tang L, Huang Q, Lee PPC, 2019. MV-Sketch: a fast and compact invertible sketch for heavy flow detection in network data streams. Proc IEEE INFOCOM Conf on Computer Communications, p.2026-2034. https://doi.org/10.1109/INFOCOM.2019.8737499
Wang F, Jiang DD, Qi S, et al., 2021. An Adaboost based link planning scheme in space-air-ground integrated networks. Mob Netw Appl, 26(2):669–680. https://doi.org/10.1007/s11036-019-01422-4
Wu Q, Chen X, Zhou Z, et al., 2021. Deep reinforcement learning with spatio-temporal traffic forecasting for data-driven base station sleep control. IEEE/ACM Trans Netw, 29(2):935–948. https://doi.org/10.1109/TNET.2021.3053771
Xu JP, Ai B, 2021. Deep reinforcement learning for handover-aware MPTCP congestion control in space-ground integrated network of railways. IEEE Wirel Commun, 28(6):200–207. https://doi.org/10.1109/MWC.001.2100116
Ya D, Bin Q, Wei N, 2021. DW-Sketch: a sketch-based scheme for realizing multi-network measurement tasks. Proc 2nd Int Conf on Computer Communication and Network Security, p.191-195. https://doi.org/10.1109/CCNS53852.2021.00043
Yang SY, Li HW, Wu Q, 2018. Performance analysis of QUIC protocol in integrated satellites and terrestrial networks. Proc 14th Int Wireless Communications & Mobile Computing Conf, p.1425-1430. https://doi.org/10.1109/IWCMC.2018.8450388
Yang WJ, Shu SJ, Cai L, et al., 2021. MM-QUIC: mobility-aware multipath QUIC for satellite networks. Proc 17th Int Conf on Mobility, Sensing and Networking, p.608-615. https://doi.org/10.1109/MSN53354.2021.00093
Yu ML, Jose L, Miao R, 2013. Software defined traffic measurement with OpenSketch. Proc 10th USENIX Conf on Networked Systems Design and Implementation, p.29-42.
Zhang R, Liu J, Yang D, et al., 2020. A survey on satellite networks based on software-defined networking. Front Data Comput, 2(3):3–17.
Author information
Authors and Affiliations
Contributions
Ziyang XING and Xiaoqiang DI designed the research. Hui QI processed the data. Ziyang XING drafted the paper. Jinyao LIU and Rui XU helped organize the paper. Jing CHEN and Ligang CONG revised the paper. Ziyang XING and Xiaoqiang DI finalized the paper.
Corresponding author
Ethics declarations
Ziyang XING, Hui QI, Xiaoqiang DI, Jinyao LIU, Rui XU, Jing CHEN, and Ligang CONG declare that they have no conflict of interest.
Additional information
Project supported by the National Natural Science Foundation of China (No. U21A20451), the Science and Technology Planning Project of Jilin Province, China (No. 20220101143JC), and the China University Industry-Academia-Research Innovation Fund (No. 2021FNA01003)
Rights and permissions
About this article
Cite this article
Xing, Z., Qi, H., Di, X. et al. A multipath routing algorithm for satellite networks based on service demand and traffic awareness. Front Inform Technol Electron Eng 24, 844–858 (2023). https://doi.org/10.1631/FITEE.2200507
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/FITEE.2200507
Key words
- Software-defined network (SDN)
- Quick user datagram protocol Internet connection (QUIC)
- Reinforcement learning
- Sketch
- Multi-service demand
- Satellite network