Abstract
Decision making in unsignalized traffic intersection can be challenging for autonomous vehicles. An autonomous system only depends on predefined street priorities and considering other cars as moving objects will cause the car to freeze and fail the maneuverer. Human drivers influence the cooperation of other drivers to evade such deadlock situations and convince others to change their behavior. Decision making processes should reason about the interaction with other drivers and anticipate a wide range of driver behaviors. In this paper, we present a reinforcement learning approach (Q-LEARNING) to learn how to interact with drivers with different situation. We enhanced the performance of Q-LEARNING algorithms by maintaining a belief over the level of cooperation of other drivers. We show that our drives successfully learns how to navigate with less deadlocks than without learning.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Li, X.-G., Gao, Z.-Y., Jia, B., et al.: Cellular automata model for unsignalized T-shaped intersection. Int. J. Mod. Phys. C 20(04), 501–512 (2009)
Fouladvand, M.E., Sadjadi, Z., Shaebani, M.R.: Characteristics of vehicular traffic flow at a roundabout. Phys. Rev. E 70(4), 046132 (2004)
Marzoug, R., Ez-Zahraouy, H., Benyoussef, A.: Simulation study of car accidents at the intersection of two roads in the mixed traffic flow. Int. J. Mod. Phys. C 26(01), 1550007 (2015)
Huang, D.W.: Modeling gridlock at roundabout. Comput. Phys. Commun. 189, 72–76 (2015)
Xie, D.F., Gao, Z.Y., Zhao, X.M., Li, K.P.: Characteristics of mixed traffic flow with non-motorized vehicles and motorized vehicles at an unsignalized intersection. Phys. A: Stat. Mech. Appl. 388(10), 2041–2050 (2009)
Yamauchi, A., Tanimoto, J., Hagishima, A., et al.: Dilemma game structure observed in traffic flow at a 2-to-1 lane junction. Phys. Rev. E 79(3), 036104 (2009)
Nakata, M., Yamauchi, A., Tanimoto, J., et al.: Dilemma game structure hidden in traffic flow at a bottleneck due to a 2 into 1 lane junction. Phys. A: Stat. Mech. Appl. 389(23), 5353–5361 (2010)
Zhang, W., Chen, W.: Dilemma game in a cellular automaton model with a non-signalized intersection. Eur. Phys. J. B 85(2), 1–8 (2012)
Chin, Y.K., Bolong, N., Kiring, A., Yang, S.S., Teo, K.T.K.: Q-learning based traffic optimization in management of signal timing plan. Int. J. Simul. Syst. Sci. Technol. 12(3), 29–35 (2011)
Balaji, P.G., German, X., Srinivasan, D.: Urban traffic signal control using reinforcement learning agents. IET Intell. Transp. Syst. 4(3), 177–188 (2010)
Abdulhai, B., Pringle, R., Karakoulas, G.J.: Reinforcement learning for true adaptive traffic signal control. J. Transp. Eng. 129(3), 278–285 (2003)
Bouderba, S.I., Moussa, N.: Reinforcement learning (Q-LEARNING) traffic light controller within intersection traffic system. In: Proceedings of the 4th International Conference on Big Data and Internet of Things, pp. 1–6 (2019)
Paissan, G., Abramson, G.: Imitation dynamics in a game of traffic. Eur. Phys. J. B 86(4), 1–6 (2013)
Nagel, K., Schreckenberg, M.: A cellular automaton model for freeway traffic. J. de physique I 2(12), 2221–2229 (1992)
Osafune, T., Takahashi, T., Kiyama, N., et al.: Analysis of accident risks from driving behaviors. Int. J. Intell. Transp. Syst. Res. 15(3), 192–202 (2007)
Bouderba, S.I., Moussa, N.: On the V2X velocity synchronization at unsignalized intersections: right hand priority based system. In: Belkasmi, M., Ben-Othman, J., Li, C., Essaaidi, M. (eds.) ACOSIS 2019. CCIS, vol. 1264, pp. 65–72. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61143-9_6
Moussa, N.: The influence of aggressive drivers on the properties of a stochastic traffic model. Eur. Phys. J. B-Condens. Matter Complex Syst. 41(3), 421–431 (2004)
Watkins, C.J., Dayan, P.: Q-learning. Mach. Learn. 8(3), 279–292 (1992)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Bouderba, S.I., Moussa, N. (2022). Dilemma Game at Unsignalized Intersection with Reinforcement Learning. In: Lazaar, M., Duvallet, C., Touhafi, A., Al Achhab, M. (eds) Proceedings of the 5th International Conference on Big Data and Internet of Things. BDIoT 2021. Lecture Notes in Networks and Systems, vol 489. Springer, Cham. https://doi.org/10.1007/978-3-031-07969-6_36
Download citation
DOI: https://doi.org/10.1007/978-3-031-07969-6_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-07968-9
Online ISBN: 978-3-031-07969-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)