Abstract
The Sim2Real gap is a topic that has been receiving a great deal of attention lately. Many Artificial Intelligence techniques, for example Reinforcement Learning, require millions of iterations to achieve satisfactory performance. This requirement often forces these techniques to solely train in simulation. If the gap between the simulated environment and the target environment is too broad, however, the trained agents will lose out on performance when deployed. Bridging this gap lowers the performance loss during deployment, in turn improving the effectiveness of these agents. This paper proposes a new technique to tackle this issue. The technique focuses on the use of demonstration samples gathered in the target environment and is based on two transfer learning fundamentals. By combining the advantages of Domain Randomization and Domain Adaptation, agents are able to transfer training performance to the target environment more successfully. Experimental results show a strong decrease in performance loss during deployment when the agent is exposed to the demonstration samples during training. The proposed technique describes a methodology that we believe can be applied in fields other than autonomous driving in order to improve transfer learning performance.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Thomas, P., Morris, A., Talbot, R., Fagerlind, H.: Identifying the causes of road crashes in Europe. Ann. Adv. Autom. Med. 57, 13 (2013)
Zhang, K., Batterman, S., Dion, F.: Vehicle emissions in congestion: comparison of work zone, rush hour and free-flow conditions. Atmos. Environ. 45, 1929–1939 (2011)
Kadian, A., et al.: Sim2real predictivity: does evaluation in simulation predict real-world performance? IEEE Rob. Autom. Lett. 5(4), 6670–6677 (2020)
Balaji, B., et al.: Deepracer: autonomous racing platform for experimentation with sim2real reinforcement learning. In: IEEE International Conference on Robotics and Automation (ICRA), 2020, pp. 2746–2754 (2020)
Wang, M., Deng, W.: Deep visual domain adaptation: a survey. Neurocomputing 312, 135–153 (2018)
Daumé III, H.: Frustratingly easy domain adaptation (2009). arXiv preprint arXiv:0907.1815
Sun, B., Feng, J., Saenko, K.: Return of frustratingly easy domain adaptation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30, no. 1 (2016)
Andrychowicz, O.M., et al.: Learning dexterous in-hand manipulation. Int. J. Rob. Res. 39(1), 3–20 (2020)
Akkaya, I., et al.: Solving rubik’s cube with a robot hand (2019). arXiv preprint arXiv:1910.07113
Hester, T., et al.: Deep q-learning from demonstrations. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1 (2018)
Levine, S., Kumar, A., Tucker, G., Fu, J.: Offline reinforcement learning: tutorial, review, and perspectives on open problems (2020). arXiv preprint arXiv:2005.01643
Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach (2010)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double q-learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30, no. 1 (2016)
Horgan, D., et al.: Distributed prioritized experience replay (2018). arXiv preprint arXiv:1803.00933
Brockman, G., et al.: Openai gym (2016)
Liang, E., et al.: RLlib: abstractions for distributed reinforcement learning. In: International Conference on Machine Learning (ICML) (2018)
Okuyama, T., Gonsalves, T., Upadhay, J.: Autonomous driving system based on deep q learnig. In: International Conference on Intelligent Autonomous Systems (ICoIAS), 2018, pp. 201–205 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Troch, A., Hoog, J.d., Vanneste, S., Balemans, D., Latré, S., Hellinckx, P. (2022). Transfer Learning in Autonomous Driving Using Real-World Samples. In: Barolli, L. (eds) Advances on P2P, Parallel, Grid, Cloud and Internet Computing. 3PGCIC 2021. Lecture Notes in Networks and Systems, vol 343. Springer, Cham. https://doi.org/10.1007/978-3-030-89899-1_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-89899-1_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-89898-4
Online ISBN: 978-3-030-89899-1
eBook Packages: EngineeringEngineering (R0)