Benchmarks: Semantic Segmentation Neural Network Verification and Objection Detection Neural Network Verification in Perceptions Tasks of Autonomous Driving

Abstract

The verification of the security of neural networks is cruicial, especially for the field of autonomous driving. Although there are currently benchmarks for the verification of the robustness of neural networks, there are hardly any benchmarks related to the field of autonomous driving, especially those related to object detection and semantic segmentation. Thus, a notable gap exists in formally verifying the robustness of semantic semantic segmentation and object detection tasks under complex, real-world conditions. To address this, we present an innovative approach to benchamark formal verification for autonomous driving perception tasks. Firstly, we propose robust verification benchmarks for semantic segmentation and object detection, supplementing existing methods. Secondly, and more significantly, we introduce a novel patch-level disturbance approach for object detection, providing a more realistic representation of real-world scenarios. By augmenting the current verification benchmarks with our novel proposals, our work contributes towards developing a more comprehensive, practical, and realistic benchmarking methodology for perception tasks in autonomous driving, thereby propelling the field towards improved safety and reliability. Our dataset and code used in this work are publicly available.

Appendices

A Network Details

In our benchmarks, we used a total of three networks, namely Unet$\_$simp, Unet$\_$upsample, and Yolo. Among them, Unet$\_$simp and Unet$\_$upsample correspond to benchmark Carvana Unet, while Yolo corresponds to benchmark CCTSDB YOLO. We have summarized the amount of parameters and the size of the models corresponding to these thress networks in the Table 2. The networks in benchmark Carvana Unet used operation such as Conv, BN, ReLu, AvgPool, ConvTranspose, etc., whereas the networks in benchmark CCTSDB YOLO used operations like Conv, BN, ReLu, MaxPool, interpolate, etc.

Table 2. Summary of the amount of parameters and model sizes of the three networks

B Implementation Details

For the benchmark Carvana Unet, we used the RMSprop optimizer, where the weight decay was set to $1 \times 10^{-8}$ and the momentum was set to 0.9. We initialized the learning rate to $1 \times 10^{-5}$, with a decay strategy of ReduceLROnPlateau, where the mode was chosen as max and patience was set to 2. We trained it for a total of 5 epochs.

For the benchmark CCTSDB YOLO, we used the SGD optimizer, where the weight decay was set to 0.0005 and the momentum was set to 0.949. We initialized the learning rate to 0.001, with a decay strategy of MultiStepLR, where the milestones was set to an array as [150, 250] and gamma was set to 0.1. We trained it for a total of 300 epochs.