Abstract
Lane changes are a vital part of vehicle motions on roads, affecting surrounding vehicles locally and traffic flow collectively. In the context of connected and automated vehicles (CAVs), this paper is concerned with the impacts of smart lane changes of CAVs on their own travel performance as well as on the entire traffic flow with the increase of the market penetration rate (MPR). On the basis of intensive microscopic traffic simulation and reinforcement learning technique, a selfish lane-changing strategy was first developed in this work to enable foresighted lane changing decisions for CAVs to improve their travel efficiency. The overall impacts of such smart lane changes on traffic flow of both CAVs and human-driven vehicles were then examined on the same simulation platform. It was found that smart lane changes were beneficial for both CAVs and the entire traffic flow, if MPR was not more than 60%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Diakaki, C., Papageorgiou, M., Papamichail, I., Nikolos, I.: Overview and analysis of vehicle automation and communication systems from a motorway traffic management perspective. Transp. Res. Part A Policy Pract. 75, 147–165 (2015)
Muratori, M., Holden, J., Lammert, M., Duran, A., Young, S., Gonder, J.: Potentials for platooning in U.S. highway freight transport. SAE Int. J. Commer. Veh. 10(1), 45–49 (2017)
Dey, K.C., et al.: A review of communication, driver characteristics, and controls aspects of cooperative adaptive cruise control (CACC). IEEE Trans. Intell. Transp. Syst. 17(2), 491–509 (2016)
Bevly, D., et al.: Lane change and merge maneuvers for connected and automated vehicles: a survey. IEEE Trans. Intell. Veh. 1(1), 105–120 (2016)
Desjardins, C., Chaib-draa, B.: Cooperative adaptive cruise control: a reinforcement learning approach. IEEE Trans. Intell. Transp. Syst. 12(4), 1248–1260 (2011)
Milanes, V., Shladover, S.E., Spring, J., Nowakowski, C., Kawazoe, H., Nakamura, M.: Cooperative adaptive cruise control in real traffic situations. IEEE Trans. Intell. Transp. Syst. 15(1), 296–305 (2014)
Amoozadeh, M., Deng, H., Chuah, C.N., Zhang, H.M., Ghosal, D.: Platoon management with cooperative adaptive cruise control enabled by VANET. Veh. Commun. 2(2), 110–123 (2015)
Ge, J.I., Orosz, G.: Dynamics of connected vehicle systems with delayed acceleration feedback. Transp. Res. Part C Emerg. Technol. 46, 46–64 (2014)
Xiao, L., Wang, M., Schakel, W., Arem, B.V.: Unravelling effects of cooperative adaptive cruise control deactivation on traffic flow characteristics at merging bottlenecks. Transp. Res. Part C Emerg. Technol. 96, 380–397 (2018)
Liu, H., Kan, X., Shladover, S.E., Lu, X.-Y., Ferlis, R.E.: Modeling impacts of cooperative adaptive cruise control on mixed traffic flow in multi-lane freeway facilities. Transp. Res. Part C Emerg. Technol. 95, 261–279 (2018)
Nilsson, J., Sjoberg, J.: Strategic decision making for automated driving on two-lane, one way roads using model predictive control. In: 2013 IEEE Intelligent Vehicles Symposium (IV), Gold Coast City, Australia, pp. 1253–1258 (2013)
You, F., Zhang, R., Lie, G., Wang, H., Wen, H., Xu, J.: Trajectory planning and tracking control for autonomous lane change maneuver based on the cooperative vehicle infrastructure system. Expert Syst. Appl. 42(14), 5932–5946 (2015)
Tehrani, H., Do, Q.H., Egawa, M., Muto, K., Yoneda, K., Mita, S.: General behavior and motion model for automated lane change. In: 2015 IEEE Intelligent Vehicles Symposium (IV) 2015, COEX, Seoul, South Korea, pp. 1154–1159 (2015)
Younes, M.B., Boukerche, A.: A vehicular network based intelligent lane change assistance protocol for highways. In: 2017 IEEE International Conference on Communications (ICC), 2017, Paris, France, pp. 1–6 (2017)
Du, Y., Wang, Y., Chan, C.Y.: Autonomous lane-change controller via mixed logical dynamical. In: IEEE 17th International Conference on Intelligent Transportation Systems (ITSC) 2014, Qingdao, China, pp. 1154–1159 (2014)
Zheng, Z.: Recent developments and research needs in modeling lane changing. Transp. Res. Part B Methodological 60, 16–32 (2014)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. The MIT Press, Cambridge (2018)
Spiliopoulou, A., Perraki, G., Papageorgiou, M., Roncoli, C.: Exploitation of ACC systems towards improved traffic flow efficiency on motorways. In 5th IEEE International Conference on Models and Technologies for Intelligent Transportation Systems: Proceedings, Naples, Italy, pp. 37–43 (2017)
Treiber, M., Hennecke, A., Helbing, D.: Congested traffic states in empirical observations and microscopic simulations. Phys. Rev. E 62(2), 1805–1824 (2000)
Acknowledgements
This work is supported in part by the National Natural Science Foundation of China (project number: 71771200) and the National Key Research and Development Program of China (project number: 2018YFB1600504; 2017YFE9134700) as well as by the European Research Council in the frame of the project TRAMAN21/ERC Advanced Grant Agreement n. 321132 under the European Union’s Seventh Framework Programme (FP/2007-2013). The authors would like to thank Prof. Bart van Arem and his group for their support in providing information related to the freeway network considered in this work.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Ye, F., Wang, L., Wang, Y., Guo, J., Papamichail, I., Papageorgiou, M. (2019). A Reinforcement Learning Approach to Smart Lane Changes of Self-driving Cars. In: Moura Oliveira, P., Novais, P., Reis, L. (eds) Progress in Artificial Intelligence. EPIA 2019. Lecture Notes in Computer Science(), vol 11804. Springer, Cham. https://doi.org/10.1007/978-3-030-30241-2_47
Download citation
DOI: https://doi.org/10.1007/978-3-030-30241-2_47
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30240-5
Online ISBN: 978-3-030-30241-2
eBook Packages: Computer ScienceComputer Science (R0)