Abstract
Artificial Intelligence is a current megatrend in computer science and almost every aspect of digitization. A scenario for training robots in a virtual environment to fulfil tasks in the real world is created to offer engineering students relevant insights into the field of neural networks. The proposed system generates training data to train a convolutional neural network (CNN) to autonomously drive a mobile robot using computer vision. The physical aspect of the setup includes the mobile robot and two different race circuits to evaluate the driving characteristics. The digital aspect consists of a 3D environment and a digital representation of the physical robot, both of which are developed using the game engine Unity. Inside the 3D environment, an infinite, procedurally generated road is created for the digital robot to drive on. The generated images from the virtual camera of the virtual robots are the basis to train the CNN to maneuver the physical robot in the real-world experiment.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cragun M.: DRIVE Sim on Omniverse. https://reg.rainfocus.com/flow/nvidia/nvidiagtc/ap2/page/sessioncatalog/session/16301039199180013WUV
State G, Wawrzyniak L.: Isaac Gym and Omniverse: High Performance Reinforcement Learning Evolved. https://reg.rainfocus.com/flow/nvidia/nvidiagtc/ap2/page/sessioncatalog/session/1628832827500001qlRX
Adrien G, Qiao W, Yohann C, Eleonora V.: Virtual worlds as Proxy for Multi-Object Tracking Analysis. arxiv:1605.06457
Ros G, Sellart L, Materzynska J, Vazquez D, Lopez AM.: The SYNTHIA dataset: a large collection of synthetic images for semantic segmentation of Urban Scenes. https://doi.org/10.1109/CVPR.2016.352
Kar A.: Meta-Sim: Learning to generate synthetic datasets. https://doi.org/10.1109/ICCV.2019.00465
Unity Technologies, Unity ML-Agents Toolkit. https://github.com/Unity-Technologies/ml-agents
Wang S, Yang J, Hu R, Qingnian Z.: Research on unmanned ship simulation on the basis of Unity3d. https://doi.org/10.1145/3232651.3232652
Miller D, Gibson S, Navarro A.: Advance your robot autonomy with ROS 2 and Unity. https://blog.unity.com/manufacturing/advance-your-robot-autonomy-with-ros-2-and-unity
Yakovlev A, Navarro A.: Simulate robots with more realism: what’s new in physics for Unity 2021.2 beta. https://blog.unity.com/technology/simulate-robots-with-more-realism-whats-new-in-physics-for-unity-20212-beta
Sita E, Horvath CM, Thomessen T, Korondi P, Pipe AG.: ROS-Unity3D based system for monitoring of an industrial robotic process. https://doi.org/10.1109/SII.2017.8279361
Berriel RF, Tabelini L, Cardoso VB, Guidolini,R.: Heading direction estimation using deep learning with automatic large-scale data acquisition. https://doi.org/10.1109/IJCNN.2018.8489154
Fischer K.: Piecewise linear approximation of Bézier curves. https://hcklbrrfnn.files.wordpress.com/2012/08/bez.pdf
Anas: URP material pack Vol 3 unity asset store. https://assetstore.unity.com/packages/vfx/shaders/urp-material-pack-vol-3-187839?aid=1101l4LJP &utm_campaign=unity_affiliate &utm_medium=affiliate &utm_source=partnerize-linkmaker
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Rolf, J., Wolf, M., Gerhard, D. (2023). Investigating the Impact of Variable Effects in Virtual Training on the Behavior of a Physical Autonomous Robot. In: Auer, M.E., El-Seoud, S.A., Karam, O.H. (eds) Artificial Intelligence and Online Engineering. REV 2022. Lecture Notes in Networks and Systems, vol 524. Springer, Cham. https://doi.org/10.1007/978-3-031-17091-1_59
Download citation
DOI: https://doi.org/10.1007/978-3-031-17091-1_59
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-17090-4
Online ISBN: 978-3-031-17091-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)