Authors:
Nelson Rodrigues
1
;
2
;
António Coelho
2
;
1
and
Rosaldo J. F. Rossetti
1
;
3
Affiliations:
1
Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias s/n, 4200-465, Porto, Portugal
;
2
INESC TEC, Rua Dr. Roberto Frias s/n, 4200-465, Porto, Portugal
;
3
LIACC - Artificial Intelligence and Computer Science Lab, Rua Dr. Roberto Frias s/n, 4200-465, Porto, Portugal
Keyword(s):
Procedural Content Generation, Inverse Procedural Modelling, Knowledge Graphs, Generative AI.
Abstract:
Driving simulators are essential tools for training, education, research, and scientific experimentation. However, the diversity and quality of virtual environments in simulations is limited by the specialized human resources availability for authoring the content, leading to repetitive scenarios and low complexity of real-world scenes. This work introduces a pipeline that can process text-based narratives outlining driving experiments to procedurally generate dynamic traffic simulation scenarios. The solution uses Retrieval-Augmented Generation alongside local open-source Large Language Models to analyse unstructured textual information and produce a knowledge graph that encapsulates the world scene described in the experiment. Additionally, a context-based formal grammar is generated through inverse procedural modelling, reflecting the game mechanics related to the interactions among the world entities in the virtual environment supported by CARLA driving simulator. The proposed pi
peline aims to simplify the generation of virtual environments for traffic simulation based on descriptions from scientific experiment, even for users without expertise in computer graphics.
(More)