Abstract
Autonomous systems (AxS) have successfully been employed in military tasks such as search and rescue, logistics, and reconnaissance. Whether it be on land, at sea, or in the air, diverse and representative training samples are indispensable for operationalising recent advances in artificial intelligence (AI). Applying deep learning to the task of armoured fighting vehicle (AFV) recognition, we examine the role synthetic data are capable of playing in training image classification and object detection models. To this end we implement a modular pipeline for the controlled generation of synthetic samples and their combination with real data with downstream building blocks for data augmentation and adversarial machine learning. This lets us conduct well-structured experiments (e.g., involving varying lighting conditions or adversarial perturbations) and develop hypotheses regarding the most beneficial composition of the training data set and the influence of individual pipeline building blocks on performance or robustness. In order to bridge the simulation-to-reality gap we use data augmentation techniques akin to domain randomisation. In particular, we fuse images with fractal pat-terns, which, in their structural complexity, resemble many forms of military camouflage. Anticipating adversarial attacks on our computer vision systems, we also train a set of more robust models by means of adversarial training, a well-studied defensive measure. Our experiments follow a rigorous evaluation protocol accounting for the multidimensional nature of both performance and robustness. We envision future applications of our thorough approach to training AxS beyond AFV recognition – in all dimensions of modern battlespace.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
In this work, we use common perturbations synonymously with corruptions of data quality.
References
Żarski, M., Wójcik, B., Miszczak, J.A., Blachowski, B., Ostrowski, M.: Computer Vision based inspection on post-earthquake with UAV synthetic dataset. IEEE Access 10, 108134–108144 (2022)
Bai, T., Luo, J., Zhao, J., Wen, B., Wang, Q.: Recent advances in adversarial training for adversarial robustness (2021)
Hendrycks, D., Dietterich, T.: Benchmarking neural network robustness to common corruptions and perturbations (2019)
Hendrycks, D., et al.: PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures (2021)
Jones, D., Snider, C., Nassehi, A., Yon, J., Hicks, B.: Characterising the digital twin: a systematic literature review. CIRP J. Manuf. Sci. Technol. 29, 36–52 (2020)
Kataoka, H., et al.: Pre-training without natural images. Int. J. Comput. Vision 130(4), 990–1007 (2022). https://doi.org/10.1007/s11263-021-01555-8
Krump, M., Ruß, M., Stütz, P.: Deep learning algorithms for vehicle detection on UAV platforms: first investigations on the effects of synthetic training. In: Mazal, J., Fagiolini, A., Vasik, P. (eds.) MESAS 2019. LNCS, vol. 11995, pp. 50–70. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-43890-6_5
Krump, M., Stütz, P.: UAV based vehicle detection on real and synthetic image pairs: performance differences and influence analysis of context and simulation parameters. In: Mazal, J., et al. (eds.) MESAS 2021. LNCS, vol. 13207, pp. 3–25. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-98260-7_1
Krump, M., Stütz, P.: UAV based vehicle detection with synthetic training: identification of performance factors using image descriptors and machine learning. In: Mazal, J., Fagiolini, A., Vasik, P., Turi, M. (eds.) MESAS 2020. LNCS, vol. 12619, pp. 62–85. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-70740-8_5
Lecun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015). https://doi.org/10.1038/nature14539
Liu, M., Fang, S., Dong, H., Xu, C.: Review of digital twin about concepts, technologies, and industrial applications. J. Manuf. Syst. 58, 346–361 (2021)
Majumdar, R., Mathur, A., Pirron, M., Stegner, L., Zufferey, D.: Paracosm: a test framework for autonomous driving simulations. In: Guerra, E., Stoelinga, M. (eds.) FASE 2021. LNCS, vol. 12649, pp. 172–195. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-71500-7_9
Nickchen, T.: Deep learning for automating additive manufacturing process chains. Ph.D. thesis (2021)
Niranjan, D.R., VinayKarthik, B.C.: Deep learning based object detection model for autonomous driving research using carla simulator, pp. 1251–1258. IEEE (2021)
Schwarzschild, A., Goldblum, M., Gupta, A., Dickerson, J., Goldstein, T.: Just how toxic is data poisoning? A unified benchmark for backdoor and data poisoning attacks. In: International Conference on Machine Learning, pp. 9389–9398 (2021)
Theiler, J.: Estimating fractal dimension. J. Opt. Soc. Am. A 7(6), 1055–1073 (1990). https://doi.org/10.1364/josaa.7.001055
Uysal, U.: Machine learning & simulation for military applications. Github (2020). https://github.com/UgurUysal86/MLS4MIL
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Liegl, C.J. et al. (2023). Simulation: The Great Enabler?. In: Mazal, J., et al. Modelling and Simulation for Autonomous Systems. MESAS 2022. Lecture Notes in Computer Science, vol 13866. Springer, Cham. https://doi.org/10.1007/978-3-031-31268-7_19
Download citation
DOI: https://doi.org/10.1007/978-3-031-31268-7_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-31267-0
Online ISBN: 978-3-031-31268-7
eBook Packages: Computer ScienceComputer Science (R0)