Abstract
To accurately test and validate algorithms used in autonomous vehicles, numerous test vehicles and very large data sets are required, resulting in safety constraints and increased financial cost. For this reason, it is desired to train the algorithms at least partly in simulation. In this work we lay the focus on the camera sensor and propose a novel methodology for injecting the instance of simulated vehicles into the camera data of real vehicles. To get qualitative results and improve generalization capabilities, the simulated data must sufficiently correspond to real-world sensor data in order to prevent loss of performance when moving the model to a real environment after training. The realism of the output is evaluated by object detection systems and a realism score produced by a CNN. Results show the potential of this approach for improving hybrid simulators for the validation of autonomous vehicles.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
de Hoog, J., Pepermans, M., Mercelis, S., Hellinckx, P.: Towards a scalable distributed real-time hybrid simulator for autonomous vehicles. In: Advances on P2P, Parallel, Grid, Cloud and Internet Computing, vol. 24, pp. 447–456. Springer International Publishing, Cham (2019)
Tsai, Y., Shen, X., Lin, Z., Sunkavalli, K., Lu, X., Yang, M.: Deep image harmonization. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2799–2807. IEEE Computer Society, Los Alamitos, July 2017
Wu, H., Zheng, S., Zhang, J., Huang, K.: GP-GAN: towards Realistic High-Resolution Image Blending. In: Proceedings of the 27th ACM International Conference on Multimedia, ser. MM 2019, pp. 2487–2495. Association for Computing Machinery, New York (2019)
Gechter, F., Dafflon, B., Gruer, P., Koukam, A.: Towards a hybrid real/virtual simulation of autonomous vehicles for critical scenarios. In: The Sixth International Conference on Advances in System Simulation (SIMUL 2014), pp. 14–17 (2014)
Reinhard, E., Adhikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Comput. Graphics Appl. 21(4), 34–41 (2001)
Sunkavalli, K., Johnson, M.K., Matusik, W., Pfister, H.: Multi-scale image harmonization. ACM Trans. Graph. 29(4), 1–10 (2010)
Tsai, Y.-H., Shen, X., Lin, Z., Sunkavalli, K., Yang, M.-H.: Sky is not the limit. ACM Trans. Graph. 35(4), 1–11 (2016)
Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Trans. Graph. 22(3), 313–318 (2003)
Xue, S., Agarwala, A., Dorsey, J., Rushmeier, H.: Understanding and improving the realism of image composites. ACM Trans. Graph. 31(4), 1–10 (2012)
Zhu, J.-Y., Krahenbuhl, P., Shechtman, E., Efros, A.A.: Learning a discriminative model for the perception of realism in composite images. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 3943–3951. IEEE, December 2015
Zhang, L., Wen, T., Shi, J.: Deep image blending. In: 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 231–240. IEEE Computer Society, Los Alamitos, March 2020
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and improving the image quality of StyleGAN. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8110–8119 (2020)
Blender Foundation: Cycles Open Source Production Rendering (2018). https://www.cycles-renderer.org/. Accessed 30 June 2020
Burt, P., Adelson, E.: The Laplacian pyramid as a compact image code. IEEE Trans. Commun. 31(4), 532–540 (1983)
Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3354–3361. IEEE, June 2012
Davis, J., Goadrich, M.: The relationship between precision-recall and roc curves. In: Proceedings of the 23rd International Conference on Machine Learning, ser. ICML 2006, pp. 233–240. Association for Computing Machinery, New York (2006)
Redmon, J., Farhadi, A.: Yolov3: an incremental improvement. arXiv preprint arXiv:1804.02767 (2018)
Dai, J., Li, Y., He, K., Sun, J.: R-FCN: object detection via region-based fully convolutional networks. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, ser. NIPS 2016, pp. 379–387. Curran Associates Inc., Red Hook (2016)
Everingham, M., Gool, L.V., Williams, C.K.I., Winn, J., Zisserman, A.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vision 88(2), 303–338 (2009)
Blender Foundation: Eevee rendering introduction (2018). https://docs.blender.org/manual/en/latest/render/eevee/introduction.html. Accessed 30 June 2020
Acknowledgements
This research received funding from the Flemish Government under the “Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen” programme.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Balemans, D., De Boeck, Y., de Hoog, J., Anwar, A., Mercelis, S., Hellinckx, P. (2021). Towards Hybrid Camera Sensor Simulation for Autonomous Vehicles. In: Barolli, L., Takizawa, M., Yoshihisa, T., Amato, F., Ikeda, M. (eds) Advances on P2P, Parallel, Grid, Cloud and Internet Computing. 3PGCIC 2020. Lecture Notes in Networks and Systems, vol 158. Springer, Cham. https://doi.org/10.1007/978-3-030-61105-7_29
Download citation
DOI: https://doi.org/10.1007/978-3-030-61105-7_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-61104-0
Online ISBN: 978-3-030-61105-7
eBook Packages: EngineeringEngineering (R0)