Object Detection to Evaluate Image-to-Image Translation on Different Road Conditions

Abstract

In vision-based navigation, image-to-image translation systems have been recently proposed to remove the effect of variable road conditions from images, in order to normalize the available data towards a common distribution (e.g. sunny condition). In this context, previous works have proposed image de-raining using generative adversarial networks (GAN) and evaluated performance based on pixel-to-pixel similarity measures that do not account for semantic content relevant for driving. Here, we propose to evaluate the performance of a de-raining GAN using an object detection neural network pretrained to find cars and pedestrians. We conducted experiments on the CARLA simulator to collect training and evaluation data under several weather conditions. Results indicate that GAN de-rained images achieve a high object detection performance in some conditions, but on average lower than object detection on the original rainy images. Future work will concentrate on improving semantic reconstruction and detection of other road elements (e.g. lanes, signs).