Abstract
Self-driving is a significant application of deep reinforcement learning. We present a deep reinforcement learning algorithm for control policies of self-driving vehicles. This method aggregates multiple sub-policies based on the deep deterministic policy gradient algorithm and centralized experience replays. The aggregated policy converges to the optimal policy by aggregating those sub-policies. It helps reduce the training time largely since each sub-policy is trained with less time. Experimental results on the open racing car simulator platform demonstrates that the proposed algorithm is able to successfully learn control policies, with a good generalization performance. This method outperforms the deep deterministic policy gradient algorithm with 56.7% less training time.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Urmson, C.: Self-driving cars and the urban challenge. IEEE Intell. Syst. 23(2), 66–68 (2008)
Krizhevsky, A., Sutskever, I., Hinton, G.: ImageNet classification with deep convolutional neural networks. In: NIPS 2012 Proceedings of the 25th International Conference on Neural Information Processing Systems, pp. 1097–1105. Curran Associates Inc., New York (2012)
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015)
Bojarski, M., Del Testa, D., Dworakowski, D., et al.: End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316 (2016)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Arulkumaran, K., Deisenroth, M.P., Brundage, M., et al.: Deep reinforcement learning: a brief survey. IEEE Signal Process. Mag. 34(6), 26–38 (2017)
Mnih, V., Kavukcuoglu, K., Silver, D., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)
Chae, H., Kang, C.M., Kim, B.D., et al.: Autonomous braking system via deep reinforcement learning. arXiv preprint arXiv:1702.02302 (2017)
Xia, W., Li, H.Y.: Training method of automatic driving strategy based on deep reinforcement learning. J. Integr. Technol. 6(3), 29–40 (2017)
Wymann, B., Espié, E., Guionneau, C., et al.: TORCS: the open racing car simulator (2015)
Loiacono, D., Cardamone, L., Lanzi, P.L.: Simulated car racing championship: competition software manual. Politecnico di Milano, Dipartimento di Elettronica, Informazione e Bioingegneria, Italy (2013)
Lillicrap, T.P., Hunt, J.J., Pritzel, A., et al.: Continuous control with deep reinforcement learning. In: ICLR (2016)
Jiang, J.: A framework for aggregation of multiple reinforcement learning algorithms. Dissertation, University of Waterloo, Waterloo, Ontario, Canada (2007)
Jiang, J., Kamel, M.S.: Aggregation of reinforcement learning algorithms. In: The 2006 IEEE International Joint Conference on Neural Networks, pp. 68–72. IEEE, Vancouver (2006)
Lowe, R., Wu, Y., Tamar, A., et al.: Multi-agent actor-critic for mixed cooperative-competitive environments. In: Advances in Neural Information Processing Systems (2017)
Silver, D., Lever, G., Heess, N., et al.: Deterministic policy gradient algorithms. In: ICML 2014 Proceedings of the 31st International Conference on International Conference on Machine Learning, pp. I-387–I-395. ICML, Beijing (2014)
Acknowledgment
This work was supported by Shenzhen Engineering laboratory on Autonomous Vehicles, NSFC 61672512, and the Shenzhen S&T Funding with Grant No. JCYJ20160510154531467.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Wu, J., Li, H. (2018). Aggregated Multi-deep Deterministic Policy Gradient for Self-driving Policy. In: Skulimowski, A., Sheng, Z., Khemiri-Kallel, S., Cérin, C., Hsu, CH. (eds) Internet of Vehicles. Technologies and Services Towards Smart City. IOV 2018. Lecture Notes in Computer Science(), vol 11253. Springer, Cham. https://doi.org/10.1007/978-3-030-05081-8_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-05081-8_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-05080-1
Online ISBN: 978-3-030-05081-8
eBook Packages: Computer ScienceComputer Science (R0)