Abstract
In this study, an automatic driving system based on end-to-end learning from simulation to reality was proposed, which integrates simulation platform, network model and on-road test platform. The purpose is to verify that the trained model can be directly deployed on the real vehicle using the dataset that is completely collected in the simulation platform. The simulation platform is implemented by calibrating the camera and environment and modeling the vehicle according to its motion characteristics. It is used to collect data and test the trained model preliminarily, where the benchmark is the deviation between the virtual vehicle location and the tracking path. The network model uses a lightweight convolution neural network with image data as input and steering prediction as output. The performance of the network model is also tested in the on-road test platform, and the criteria is the proportion of the vehicle’s autonomous driving time during the whole driving. The results show that the path deviation of simulation test is less than 10 pixels, corresponding to about 5 cm on the real road. And the proportion of automatic driving time is more than 98% in the on-road test. The evaluation results can meet the automatic driving requirements. Collecting large amount of synthesized image and calculating the theoretical steering in the simulation platform can improve the certainty and accuracy of model output, and reduce the labor cost. In addition, the end-to-end lightweight network model reduces the demand for computing resources and sensor hardware. On the indoor or park scenes with structured roads as the main elements, this automatic driving system can promote the large-scale use of logistics vehicles or transport vehicles.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Scott, P., Hans, A., Xinxin, D., et al.: Perception, planning, control, and coordination for autonomous vehicles. Machines 5(1), 6 (2017)
Lei, T., Liu, M.: Deep-learning in mobile robotics - from perception to control systems: a survey on why and why not. J. Latex Class Files 14(8) (2016)
Pomerleau, D.A.: Alvinn: an autonomous land vehicle in a neural network. In: Advances in Neural Information Processing Systems 1, pp. 305–313, Colorado, USA (1988)
Jochem, T., Pomerleau, D.: Vision-based neural network road and intersection detection and traversal. In: IEEE Conference on Intelligent Robots and Systems (1995)
Lecun, Y., Muller, U., Ben, J., et al.: Off-road obstacle avoidance through end-to-end learning. In: International Conference on Neural Information Processing Systems, vol. 1, pp. 305–313 (2005)
Bojarski, M., Del Testa, D., Dworakowski, D., et al.: End to end learning for self-driving cars. Preprint at https://arxiv.org/abs/1604.07316 (2016)
Bojarski, M., Yeres, P., Choromanska, A., et al.: Explaining how a deep neural network trained with end-to-end learning steers a car. arXiv:1704.07911 (2017)
Xu, H., Gao, Y., Yu, F., Darrell, T.: End-to-end learning of driving models from large-scale video datasets. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2017)
Yang, Z., Zhang, Y., Yu, J., et al.: End-to-end multi-modal multi-task vehicle control for self-driving cars with visual perceptions (2018)
Kocic, J., Jovicic, N., Drndarevic, V.: An end-to-end deep neural network for autonomous driving designed for embedded automotive platforms. Sensors (2019). https://doi.org/10.3390/s19092064
Pfeiffer, M., Schaeuble, M., Nieto, J., et al.: From perception to decision: a data-driven approach to end-to-end motion planning for autonomous ground robots. In: IEEE International Conference on Robotics and Automation (2017)
Patel, N., Choromanska, A., Krishnamurthy, P., et al.: Sensor modality fusion with CNNs for UGV autonomous driving in indoor environments. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (2017)
Sallab, A., Abdou, M., Perot, E., et al.: Deep reinforcement learning framework for autonomous driving. Electron. Imaging 19, 70–76 (2017)
Codevilla, F., Müller, M., López, A., et al.: End-to-end driving via conditional imitation learning, pp. 1–9 (2018)
Dosovitskiy, A., Ros, G., Codevilla, F., et al.: CARLA: an open urban driving simulator. In: Conference on Robot Learning (CoRL), Mountain View, California, pp. 1–16 (2017)
Chen, C., Seff, A., Kornhauser, A., et al.: DeepDriving: learning affordance for direct perception in autonomous driving. In: International Conference on Computer Vision (ICCV) (2015)
Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zhang, Y., Wang, B., Li, J., Yu, Y. (2021). A Simulation-to-Real Autonomous Driving System Based on End-to-End Learning. In: Sun, F., Liu, H., Fang, B. (eds) Cognitive Systems and Signal Processing. ICCSIP 2020. Communications in Computer and Information Science, vol 1397. Springer, Singapore. https://doi.org/10.1007/978-981-16-2336-3_54
Download citation
DOI: https://doi.org/10.1007/978-981-16-2336-3_54
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-2335-6
Online ISBN: 978-981-16-2336-3
eBook Packages: Computer ScienceComputer Science (R0)