Abstract
We propose an object detection system for maritime search and rescue as a benchmark problem for verification of neural networks (VNN). The model to be verified is a YOLO (You Only Look Once) deep neural network for object detection and classification and has a very high number of learnable parameters (millions). We describe the workflow for defining and generating robustness properties in the regions of interest of the images, i.e., in the neighborhood of the objects to be detected by the neural network. This benchmark can be used to assess the applicability and the scalability of existing VNN tools for perception systems based on deep learning.
URL. Benchmark materials, such as trained models (.onnx), examples of properties (.vnnlib), test images, and property generation procedures, are available at https://github.com/loonwerks/vnncomp2023.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
SeaDronesSee dataset is used on a license.
- 3.
We note that in future work it may be possible to also identify more meaningful perturbations, such as changing the colors of certain objects in the image (e.g., life jackets from red to blue). Such modifications may require additional image processing to precisely identify the pixels to apply perturbation to, which brings additional challenges to be solved.
- 4.
Resizing is performed via OpenCV resize() command (bilinear interpolation).
References
Avalon - Aerial and vision-based assistance system for real time object detection in search and rescue missions. https://uni-tuebingen.de/fakultaeten/mathematisch-naturwissenschaftliche-fakultaet/fachbereiche/informatik/lehrstuehle/kognitive-systeme/projects/avalon/
SeaDronesSee dataset. https://seadronessee.cs.uni-tuebingen.de/dataset
EASA and Collins Aerospace: Formal Methods use for Learning Assurance (ForMuLA). Technical report (2023)
European Union Aviation Safety Agency (EASA): Concept Paper: Guidance for Level 1 &2 Machine Learning Applications. Concept paper for consultation (2023)
Varga, L.A., Kiefer, B., Messmer, M., Zell, A.: Seadronessee: a maritime benchmark for detecting humans in open water. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2260–2270 (2022)
Acknowledgement
The authors wish to thank Benjamin Kiefer et al. from the University of Tuebingen for the publication of the SeaDronesSee dataset, which has been used as a baseline in this research work.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kirov, D., Rollini, S.F., Chandrahas, R., Chandupatla, S.R., Sawant, R. (2024). Benchmark: Object Detection for Maritime Search and Rescue. In: Steffen, B. (eds) Bridging the Gap Between AI and Reality. AISoLA 2023. Lecture Notes in Computer Science, vol 14380. Springer, Cham. https://doi.org/10.1007/978-3-031-46002-9_19
Download citation
DOI: https://doi.org/10.1007/978-3-031-46002-9_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-46001-2
Online ISBN: 978-3-031-46002-9
eBook Packages: Computer ScienceComputer Science (R0)