Abstract
This article investigates the feasibility of implementing a reinforcement learning agent able to plan the trajectory of a simple automated vehicle 2D model in a motorway simulation. The goal is to use it to implement a non-player vehicle in serious games for driving. The agent extends a Deep Q Learning agent developed by Eduard Leurent in Stable Baselines by adding rewards in order to better meet the traffic laws. The motorway environment was enhanced as well, in order to increase realism. A multilayer perceptron model, processing cinematic inputs from the ego and other vehicles, was tested in different traffic conditions and outperformed the original model and other policies such as a heuristic and a minimal-reward one. Our experience stresses the importance of defining episode metrics to assess agent behavior, keeping into accounts factors related to safety (e.g., keeping a safe time to collision) and consumption (e.g., limiting accelerations and decelerations). This is key to define rewards and penalties able to properly train the model to meet the traffic laws while keeping a high-speed performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Massoud, R., Berta, R., Poslad, S., De Gloria, A., Bellotti, F.: IoT sensing for reality-enhanced serious games, a fuel-efficient drive use case. Sensors. 21(10), 3559 (2021). https://doi.org/10.3390/s21103559
Renault group, The Good Drive, a serious game to learning to drive, available online at: https://www.renaultgroup.com/en/news-on-air/news/the-good-drive-a-serious-game-for-learning-to-drive/
Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V.: CARLA: An Open Urban driving simulator. In: 1st Conference on Robot Learning (CoRL 2017). Mountain View, USA (2017)
Carla community: Carla simulator. Version 0.9.11. https://carla.readthedocs.io/en/latest/. Accessed 23 April 2021
Leurent, E.: An environment for autonomous driving decision-making, github repository, Available at: https://github.com/eleurent/highway-env (2018)
Gounaridou, A., Siamtanidou, E., Dimoulas, C.: A serious game for mediated education on traffic behavior and safety awareness. Educ. Sci. 11, 127 (2021). https://doi.org/10.3390/educsci11030127
Likitweerawong, K., Palee, P.: The virtual reality serious game for learning driving skills before taking practical test. In: 2018 International Conference on Digital Arts, Media and Technology (ICDAMT), pp. 158–161 (2018). https://doi.org/10.1109/ICDAMT.2018.8376515.
Hrimech, H., et al.: The effects of the use of serious game in eco-driving training. Front. ICT (2016). https://doi.org/10.3389/fict.2016.00022
Massoud, R., Poslad, S., Bellotti, F., Berta, R., Mehran, K., De Gloria, A.: A fuzzy logic module to estimate a driver’s fuel consumption for reality-enhanced serious games. Int. J. Serious Games 5(4), 45–62 (2018). https://doi.org/10.17083/ijsg.v5i4.266
Lazzaroni, L., Mazzara, A., Bellotti, F., De Gloria, A., Berta, R.: Employing an IoT framework as a generic serious games analytics engine. In: Marfisi-Schottman, I., Bellotti, F., Hamon, L., Klemke, R. (eds.) GALA 2020. LNCS, vol. 12517, pp. 79–88. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-63464-3_8
Aldape, P., Sowell, S.: Reinforcement learning for a simple racing game. https://web.stanford.edu/class/aa228/reports/2018/final150.pdf/ Retrieved 14 Sept 2020
Westera, W., et al.: Artificial intelligence moving serious gaming: presenting reusable game AI components. Educ. Inf. Technol. 25(1), 351–380 (2019). https://doi.org/10.1007/s10639-019-09968-2
Geron, A.: Hands-On Machine Learning With Scikit-Learn and Tensorflow. O’Reilly (2017)
Stable baselines: Available online at: https://stable-baselines.readthedocs.io
Tensorflow: Available online at: https://www.tensorflow.org/
Wang, Z., Schaul, T., Hessel, M., Van Hasselt, H., Lanctot, M., De Freitas, N.: Dueling network architectures for deep reinforcement learning. In: Proceedings of the 33rd International Conference on International Conference on Machine Learning, vol. 48 (ICML’16). JMLR.org (2016)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015). https://doi.org/10.1038/nature14236
Leurent, E., Mercat, J.: Social attention for autonomous decision-making in dense traffic. In: Machine Learning for Autonomous Driving Workshop at the Thirty-third Conference on Neural Information Processing Systems (NeurIPS 2019). Montreal, Canada arXiv:1911.12250 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Campodonico, G. et al. (2021). Adapting Autonomous Agents for Automotive Driving Games. In: de Rosa, F., Marfisi Schottman, I., Baalsrud Hauge, J., Bellotti, F., Dondio, P., Romero, M. (eds) Games and Learning Alliance. GALA 2021. Lecture Notes in Computer Science(), vol 13134. Springer, Cham. https://doi.org/10.1007/978-3-030-92182-8_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-92182-8_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92181-1
Online ISBN: 978-3-030-92182-8
eBook Packages: Computer ScienceComputer Science (R0)