Authors:
Stephan Pareigis
;
Daniel Riege
and
Tim Tiedemann
Affiliation:
Department of Computer Science, HAW Hamburg, Berliner Tor 7, 20099 Hamburg, Germany
Keyword(s):
Reinforcement Learning, Digital Twin, Autonomous Driving, Sim-to-Real Gap, Miniature Autonomy, Real-World Reinforcement Learning.
Abstract:
An experimental setup and preliminary validation of a platform for sim-to-real transfer in reinforcement learning for autonomous driving is presented. The platform features a 1:87 scale miniature autonomous vehicle, the tinycar, within a detailed miniature world that includes urban and rural settings. Key components include a simulation for training machine learning models, a digital twin with a tracking system using overhead cameras, an automatic repositioning mechanism of the miniature vehicle to reduce human intervention when training in the real-world, and an encoder based approach for reducing the state space dimension for the machine learning algorithms. The tinycar is equipped with a steering servo, DC motor, front-facing camera, and a custom PCB with an ESP32 micro-controller. A custom UDP-based network protocol enables real-time communication. The machine learning setup uses semantically segmented lanes of the streets as an input. These colored lanes can be directly produced
by the simulation. In the real-world a machine learning based segmentation method is used to achieve the segmented lanes. Two methods are used to train a controller (actor): Imitation learning as a supervised learning method in which a Stanley controller serves as a teacher. Secondly, Twin Delayed Deep Deterministic Policy Gradient (TD3) is used to minimize the Cross-Track Error (CTE) of the miniature vehicle with respect to its lateral position in the street. Both methods are applied equally in simulation and in the real-world and are compared. Preliminary results show high accuracy in lane following and intersection navigation in simulation and real-world, supported by precise real-time feedback from the tracking system. While full integration of the RL model is ongoing, the presented results show the platform’s potential to further investigate the sim-to-real aspects in autonomous driving.
(More)