Abstract
In Internet of Vehicles (IoV), beam tracking for multiple vehicles is a challenging topic due to the nonlinear mobility and inter-vehicle interference (IVI). This paper considers the scenario that multiple vehicles with high mobility are periodically served by the radiated beams of millimeter wave (mmWave) massive multiple-input multiple-output (MIMO) systems. The main objective is to maximize the probability of successful information transmission in each beam tracking period, where successful transmission is defined by signal-to-interference-plus-noise ratio (SINR) exceeding a threshold. Based on deep reinforcement learning, we propose a position prediction and joint selection (PPJS) scheme for beam selection of multiple vehicles in consideration of both the coverage and IVI. On one hand, long short-term memory (LSTM) network is employed to predict the future trajectory in upcoming beam tracking period for providing better beam coverage. On the other hand, multi-layer perception (MLP) network is designed to select the served beams by taking into account the IVI, where the vehicles are divided into clusters and the objective of beam tracking in each cluster is decomposed to reduce the scheme complexity. Simulation results demonstrate that the proposed PPJS scheme performs better than both the traditional position-based algorithm and deep Q-learning (DQN) algorithm.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bujari, A., Gottardo, J., Palazzi, C.E., Ronzani, D.: Message dissemination in urban IoV. In: 2019 IEEE/ACM 23rd International Symposium on Distributed Simulation and Real Time Applications (DS-RT), pp. 1–4 (2019). https://doi.org/10.1109/DS-RT47707.2019.8958708
Hou, Z., She, C., Li, Y., Zhuo, L., Vucetic, B.: Prediction and communication co-design for ultra-reliable and low-latency communications. IEEE Trans. Wireless Commun. 19(2), 1196–1209 (2020). https://doi.org/10.1109/TWC.2019.2951660
Bencivenni, C., Gustafsson, M., Haddadi, A., Zaman, A.U., Emanuelsson, T.: 5G mmWave beam steering antenna development and testing. In: 2019 13th European Conference on Antennas and Propagation (EuCAP), pp. 1–4 (2019)
Ali, M.Y., Hossain, T., Mowla, M.M.: A trade-off between energy and spectral efficiency in massive MIMO 5G system. In: 2019 3rd International Conference on Electrical, Computer Telecommunication Engineering (ICECTE), pp. 209–212 (2019). https://doi.org/10.1109/ICECTE48615.2019.9303551
Raza, A., Junaid Nawaz, S., Wyne, S., Ahmed, A., Javed, M.A., Patwary, M.N.: Spatial modeling of interference in inter-vehicular communications for 3-D volumetric wireless networks. IEEE Access 8, 108281–108299 (2020). https://doi.org/10.1109/ACCESS.2020.3001052
Zhao, L., Zhao, H., Zheng, K., Xiang, W.: Massive MIMO in 5G Networks: Selected Applications. Springer, Heidelberg (2018). https://doi.org/10.1007/978-3-319-68409-3
Zheng, K., Zhao, L., Mei, J., Shao, B., Xiang, W., Hanzo, L.: Survey of large-scale MIMO systems. IEEE Commun. Surv. Tutor. 17(3), 1738–1760 (2015). https://doi.org/10.1109/COMST.2015.2425294
Va, V., Vikalo, H., Heath, R.W.: Beam tracking for mobile millimeter wave communication systems. In: 2016 IEEE Global Conference on Signal and Information Processing (GlobalSIP), pp. 743–747 (2016). https://doi.org/10.1109/GlobalSIP.2016.7905941
Liu, Y., Jiang, Z., Zhang, S., Xu, S.: Deep reinforcement learning-based beam tracking for low-latency services in vehicular networks. In: ICC 2020–2020 IEEE International Conference on Communications (ICC), pp. 1–7 (2020). https://doi.org/10.1109/ICC40277.2020.9148759
Jeong, J., Lim, S.H., Song, Y., Jeon, S.W.: Online learning for joint beam tracking and pattern optimization in massive MIMO systems. In: IEEE INFOCOM 2020 - IEEE Conference on Computer Communications, pp. 764–773 (2020). https://doi.org/10.1109/INFOCOM41043.2020.9155475
Tran, X.V., Pham, V.H.: An analytical method for calculating the limitation of beam scanning in uniform linear array (ULA). In: 2009 International Conference on Advanced Technologies for Communications, pp. 257–260 (2009). https://doi.org/10.1109/ATC.2009.5349450
Cvetek, D., Muštra, M., Jelušić, N., Abramović, B.: Traffic flow forecasting at micro-locations in urban network using bluetooth detector. In: 2020 International Symposium ELMAR, pp. 57–60 (2020). https://doi.org/10.1109/ELMAR49956.2020.9219023
Dai, G., Ma, C., Xu, X.: Short-term traffic flow prediction method for urban road sections based on space-time analysis and GRU. IEEE Access 7, 143025–143035 (2019). https://doi.org/10.1109/ACCESS.2019.2941280
Abadi, A., Rajabioun, T., Ioannou, P.A.: Traffic flow prediction for road transportation networks with limited traffic data. IEEE Trans. Intell. Transp. Syst. 16(2), 653–662 (2015). https://doi.org/10.1109/TITS.2014.2337238
Acknowledgment
This work was supported in part by the China Nature Science Funding under Grant 61731004.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Cheng, B., Zhao, L., He, Z., Zhang, P. (2022). A Beam Tracking Scheme Based on Deep Reinforcement Learning for Multiple Vehicles. In: Gao, H., Wun, J., Yin, J., Shen, F., Shen, Y., Yu, J. (eds) Communications and Networking. ChinaCom 2021. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 433. Springer, Cham. https://doi.org/10.1007/978-3-030-99200-2_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-99200-2_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-99199-9
Online ISBN: 978-3-030-99200-2
eBook Packages: Computer ScienceComputer Science (R0)