Abstract
Making human-like decisions for autonomous driving in interactive scenarios is crucial and difficult, requiring the self-driving vehicle to reason about the reactions of interactive vehicles to its behavior. To handle this challenge, we provide an integrated prediction and planning (PnP) decision-making approach. A reactive trajectory prediction model is developed to predict the future states of other actors in order to account for the interactive nature of the behaviors. Then, n-step temporal-difference search is used to make a tactical decision and plan the tracking trajectory for the self-driving vehicle by combining the value estimation network with the reactive prediction model. The proposed PnP method is evaluated using the CARLA simulator, and the results demonstrate that PnP obtains superior performance compared to popular model-free and model-based reinforcement learning baselines.
Z. Xia—Independent Researcher.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chauhan, N.S., Kumar, N.: Traffic flow forecasting using attention enabled Bi-LSTM and GRU hybrid model. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds.) Neural Information Processing, ICONIP 2022. CCIS, vol. 1794, pp. 505–517. Springer, Singapore (2023). https://doi.org/10.1007/978-981-99-1648-1_42
Chen, C., Hu, S., Nikdel, P., Mori, G., Savva, M.: Relational graph learning for crowd navigation. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 10007–10013. IEEE (2020)
Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V.: CARLA: an open urban driving simulator. In: Conference on Robot Learning, pp. 1–16. PMLR (2017)
Gu, J., Sun, C., Zhao, H.: DenseTNT: end-to-end trajectory prediction from dense goal sets. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15303–15312 (2021)
Guo, Y., Zhang, Q., Wang, J., Liu, S.: Hierarchical reinforcement learning-based policy switching towards multi-scenarios autonomous driving. In: 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2021)
Hafner, D., Lillicrap, T., Ba, J., Norouzi, M.: Dream to control: Learning behaviors by latent imagination. In: International Conference on Learning Representations (2019)
Hafner, D., Lillicrap, T.P., Norouzi, M., Ba, J.: Mastering Atari with discrete world models. In: International Conference on Learning Representations (2020)
Hagedorn, S., Hallgarten, M., Stoll, M., Condurache, A.: Rethinking integration of prediction and planning in deep learning-based automated driving systems: a review. arXiv preprint arXiv:2308.05731 (2023)
Li, D., Zhao, D., Zhang, Q., Chen, Y.: Reinforcement learning and deep learning based lateral control for autonomous driving [application notes]. IEEE Comput. Intell. Mag. 14(2), 83–98 (2019)
Liu, J., Zeng, W., Urtasun, R., Yumer, E.: Deep structured reactive planning. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 4897–4904. IEEE (2021)
Liu, Y., Gao, Y., Zhang, Q., Ding, D., Zhao, D.: Multi-task safe reinforcement learning for navigating intersections in dense traffic. J. Franklin Inst. (2022)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Palmal, S., Arya, N., Saha, S., Tripathy, S.: A multi-modal graph convolutional network for predicting human breast cancer prognosis. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds.) Neural Information Processing, ICONIP 2022. Communications in Computer and Information Science, vol. 1794, pp. 187–198. Springer, Singapore (2023). https://doi.org/10.1007/978-981-99-1648-1_16
Silver, D., et al.: Mastering the game of go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)
Silver, D., Sutton, R.S., Müller, M.: Temporal-difference search in computer go. Mach. Learn. 87, 183–219 (2012)
Wang, J., Zhang, Q., Zhao, D.: Highway lane change decision-making via attention-based deep reinforcement learning. IEEE/CAA J. Automatica Sinica 9(3), 567–569 (2021)
Wang, J., Zhang, Q., Zhao, D.: Dynamic-horizon model-based value estimation with latent imagination. IEEE Trans. Neural Netw. Learn. Syst. (2022)
Wang, J., Zhang, Q., Zhao, D., Chen, Y.: Lane change decision-making through deep reinforcement learning with rule-based constraints. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–6. IEEE (2019)
Wen, J., Zhao, Z., Cui, J., Chen, B.M.: Model-based reinforcement learning with self-attention mechanism for autonomous driving in dense traffic. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds.) Neural Information Processing, ICONIP 2022. LNCS, vol. 13624, pp. 317–330. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-30108-7_27
Wu, J., Huang, Z., Lv, C.: Uncertainty-aware model-based reinforcement learning: methodology and application in autonomous driving. IEEE Trans. Intell. Veh. 8, 194–203 (2022)
Zhao, X., Chen, Y., Guo, J., Zhao, D.: A spatial-temporal attention model for human trajectory prediction. IEEE CAA J. Autom. Sinica 7(4), 965–974 (2020)
Acknowledgements
This work was supported by the National Key Research and Development Program of China under Grants 2022YFA1004000, National Natural Science Foundation of China (NSFC) under Grants No. 62173325 and CCF Baidu Open Fund.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Liu, X., Zhang, Q., Gao, Y., Xia, Z. (2024). PnP: Integrated Prediction and Planning for Interactive Lane Change in Dense Traffic. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Lecture Notes in Computer Science, vol 14452. Springer, Singapore. https://doi.org/10.1007/978-981-99-8076-5_22
Download citation
DOI: https://doi.org/10.1007/978-981-99-8076-5_22
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8075-8
Online ISBN: 978-981-99-8076-5
eBook Packages: Computer ScienceComputer Science (R0)