Zheyu Cui
ABSTRACT
Aiming at the problem of driving scenarios redundancy in the lane change decision-making, this paper proposes a deep reinforcement learning method (DRL) for lane change decision based on embedded attention mechanism (CADQN). The algorithm introduces the Convolutional Attention Mechanism Module (CBAM) into the DQN network to optimize the scenarios in time and space dimensions, and assist connected vehicles in making lane changing decisions. The algorithm is verified by the traffic simulation platform under the highway environment, and the results show that CADQN is helpful to improve the global traffic efficiency, and with the increase of traffic flow density, the benefit is more significant. Moreover, the visualization results of the attention layer in the CADQN can guide the optimization of the driving scenario.
- L Li, R Jiang, B Jia and X M Zhao, 2010. Modern Traffic Flow Theory and Applications, Vol. I, Highway Traffic. Tsinghua University Press, Beijing, China. ISBN 978-7-302-23807-2 (in Chinese).Google Scholar
- Cassidy, M. J., & Bertini, R. L. (1999). Some traffic features at freeway bottlenecks. Transportation Research Part B: Methodological, 33(1), 25-42.Google ScholarCross Ref
- R. L. Bertini and M. T. Leal (2005). Empirical study of traffic features at a freeway lane drop. Journal of Transportation Engineering, 131(6), 397-407.Google ScholarCross Ref
- Y. Hou, P. Edara and C. Sun (2013). Modeling mandatory lane changing using Bayes classifier and decision trees. IEEE Transactions on Intelligent Transportation Systems, 15(2), 647-655.Google ScholarCross Ref
- J. A. Laval and C. F. Daganzo (2006). Lane-changing in traffic streams. Transportation Research Part B: Methodological, 40(3), 251-264.Google ScholarCross Ref
- J. C Munoz and C. F. Daganzo (2002). Moving bottlenecks: a theory grounded on experimental observation. Emerald Group Publishing Limited.Google Scholar
- N. H. Gartner, P. J. Tarnoff, and C. M. Andrews (1991). Evaluation of optimized policies for adaptive control strategy. Transportation Research Record, (1324).Google Scholar
- M. Brackstone,, M. McDonald and J. Wu (1998). Lane changing on the motorway: Factors affecting its occurrence, and their implications.Google Scholar
- J. Lygeros, D. N. Godbole and S. Sastry (1998). Verified hybrid controllers for automated vehicles. IEEE transactions on automatic control, 43(4), 522-539.Google ScholarCross Ref
- K. Nagel, D. E. Wolf, P. Wagner and P Simon (1998). Two-lane traffic rules for cellular automata: A systematic approach. Physical Review E, 58(2), 1425.Google ScholarCross Ref
- S. Maerivoet and B. De Moor (2005). Cellular automata models of road traffic. Physics reports, 419(1), 1-64.Google Scholar
- A. Eidehall, J. Pohl, F. Gustafsson and J. Ekmark (2007). Toward autonomous collision avoidance by steering. IEEE Transactions on Intelligent Transportation Systems, 8(1), 84-94.Google ScholarDigital Library
- D. D. Salvucci, H. M. Mandalia, N. Kuge, and T. Yamamura. (2007). Lane-change detection using a computational driver model. Human factors, 49(3), 532-542.Google Scholar
- A. Doshi, and M. Trivedi (2008). A comparative exploration of eye gaze and head motion cues for lane change intent prediction. In 2008 IEEE Intelligent Vehicles Symposium (pp. 49-54). IEEE.Google ScholarCross Ref
- R. J.Kiefer and J. M. Hankey (2008). Lane change behavior with a side blind zone alert system. Accident Analysis & Prevention, 40(2), 683-690.Google ScholarCross Ref
- L S Jin, W P Fang, Y N Zhang, S B Yang and H J Hou (2009). Research on safety lane change model of driver assistant system on highway. In 2009 IEEE Intelligent Vehicles Symposium (pp. 1051-1056). IEEE.Google Scholar
- P. G. Gipps (1986). A model for the structure of lane-changing decisions. Transportation Research Part B: Methodological, 20(5), 403-414.Google ScholarCross Ref
- P. G. Gipps (1981). A behavioural car-following model for computer simulation. Transportation Research Part B: Methodological, 15(2), 105-111.Google ScholarCross Ref
- K. Ahmed, B. M. Akiva, H. Koutsopoulos, and R. Mishalani (1996). Models of freeway lane changing and gap acceptance behavior. Transportation and traffic theory, 13, 501-515.Google Scholar
- K. I. Ahmed, (1999). Modeling drivers' acceleration and lane changing behavior (Doctoral dissertation, Massachusetts Institute of Technology).Google Scholar
- J. G. Hunt and G. D. Lyons (1994). Modelling dual carriageway lane changing using neural networks. Transportation Research Part C: Emerging Technologies, 2(4), 231-245.Google ScholarCross Ref
- A. Dumbuya, A. Booth, N. Reed, A. Kirkham, T. Philpott, J. Zhao and R. Wood (2009). Complexity of traffic interactions: Improving behavioural intelligence in driving simulation scenarios. In Complex systems and self-organization modelling (pp. 201-209), Springer, Berlin, Heidelberg.Google ScholarCross Ref
- X. Li, X. Xu and L. Zuo (2015). Reinforcement learning based overtaking decision-making for highway autonomous driving. In 2015 Sixth International Conference on Intelligent Control and Information Processing (ICICIP) (pp. 336-342). IEEE.Google ScholarCross Ref
- Y. Li, (2017). Deep reinforcement learning: An overview. arXiv preprint arXiv:1701.07274.Google Scholar
- V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, ... and D. Hassabis (2015). Human-level control through deep reinforcement learning. nature, 518(7540), 529-533.Google Scholar
- S. Woo, J. Park, J. Y. Lee and I. S. Kweon (2018). Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV) (pp. 3-19).Google ScholarDigital Library
- E. Leurent (2018) An environment for autonomous driving decision-making. https://github.com/eleurent/highway-env.Google Scholar
- M. Treiber, A. Hennecke and D. Helbing (2000). Congested traffic states in empirical observations and microscopic simulations. Physical review E, 62(2), 1805.Google Scholar
Index Terms
- An Intelligent Lane Changing Decision Method for Connected Vehicles
Recommendations
Reliable and Efficient Lane Changing Behaviour for Connected Autonomous Vehicle through Deep Reinforcement Learning
AbstractThe establishment of future intelligent transport systems is dependable on the reliable and seamless function of Connected and Autonomous Vehicles (CAV). Reinforcement learning (RL), which allows autonomous vehicles (AVs) to learn an ideal driving ...
Development of a methodology to demonstrate the environmental impact of connected vehicles under lane-changing conditions
This paper describes a lane-change model for connected vehicles, and evaluates the environmental impact per road level of service (LOS) when a host vehicle makes a lane change. During manual driving, the host vehicle accelerates or decelerates to create ...
A deep reinforcement learning-based approach for autonomous lane-changing velocity control in mixed flow of vehicle group level
AbstractAs an important driving behavior, lane-changing has a great impact on the safety and efficiency of traffic flow interacting with surrounding vehicles, especially in mixed traffic flows with autonomous vehicles and human-driven vehicles. This ...
Comments