Skip to main content
Springer Nature Link
Log in

Deep Learning Algorithms for Vehicle Detection on UAV Platforms: First Investigations on the Effects of Synthetic Training

  • Conference paper
  • First Online:
Modelling and Simulation for Autonomous Systems (MESAS 2019)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11995))

Abstract

Vehicle detection on aerial imagery taken with UAVs (unmanned aerial vehicles) plays an important role in many fields of application, such as traffic monitoring, surveillance or defense and rescue missions. Deep learning based object detectors are often used to overcome the resulting detection challenges. The generation of training data under different conditions and with the necessary variance is difficult and costly in real life. Therefore, virtual simulation environments are meanwhile often applied for this purpose. Our current research interests focus on the difference in performance, also called reality gap, of trainable vehicle detectors between both domains and the influence of differently designed training data. A general method for automatic image annotation with the required bounding boxes is described. In the first part of the investigations the training behavior of YOLOv3 on the natural UAVDT data set is analyzed and examined to what extent algorithms trained with natural images can be evaluated in the simulation. Finally, it is shown which performance can be achieved by exclusively synthetic training and how the performance can be improved by synthetic extension of the natural training set.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Du, D., et al.: The unmanned aerial vehicle benchmark: object detection and tracking. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11214, pp. 375–391. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01249-6_23

    Chapter  Google Scholar 

  2. Li, Q., Mou, L., Xu, Q., Zhang, Y., Zhu, X.X.: R3-Net: a deep network for multi-oriented vehicle detection in aerial images and videos. IEEE Geosci. Remote Sens. Soc., 1–14 (2019)

    Google Scholar 

  3. Radovic, M., Adarkwa, O., Wang, Q.: Object recognition in aerial images using convolutional neural networks. J. Imaging 3(2), 21 (2017)

    Article  Google Scholar 

  4. Xu, Y., Yu, G., Wang, Y., Wu, X., Ma, Y.: Car detection from low-altitude UAV imagery with the faster R-CNN. J. Adv. Transp. 2017, 1–10 (2017)

    Google Scholar 

  5. Tayara, H., Soo, K.G., Chong, K.T.: Vehicle detection and counting in high-resolution aerial images using convolutional regression neural network. IEEE Access 6, 2220–2230 (2017)

    Article  Google Scholar 

  6. Tang, T., Deng, Z., Zhou, S., Lei, L., Zou, H.: Fast vehicle detection in UAV images. In: RSIP 2017 - International Workshop on Remote Sensing with Intelligent Processing, Proceedings, pp. 1–5 (2017)

    Google Scholar 

  7. Benjdira, B., Khursheed, T., Koubaa, A., Ammar, A., Ouni, K.: Car detection using unmanned aerial vehicles: comparison between faster R-CNN and YOLOv3. In: 1st International Conference on Unmanned Vehicle Systems-Oman, UVS 2019, pp. 1–6 (2019)

    Google Scholar 

  8. Li, W., Li, H., Wu, Q., Chen, X., Ngan, K.N.: Simultaneously detecting and counting dense vehicles from drone images. IEEE Trans. Ind. Electron. 66(12), 9651–9662 (2019)

    Article  Google Scholar 

  9. Lu, J., et al.: A vehicle detection method for aerial image based on YOLO. J. Comput. Commun. 06, 98–107 (2018)

    Article  Google Scholar 

  10. Lechgar, H., Bekkar, H., Rhinane, H.: Detection of cities vehicle fleet using YOLO V2 and aerial images. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. ISPRS Arch. 42, 121–126 (2019)

    Article  Google Scholar 

  11. Ferwerda, J.A.: Three varieties of realism in computer graphics. In: Rogowitz, B.E., Pappas, T.N. (eds.) Proceedings SPIE Human Vision and Electronic (2003)

    Google Scholar 

  12. Presagis - COTS Modelling and Simulation Software. https://www.presagis.com/en/. https://www.presagis.com/en/page/academic-programs/

  13. Redmon, J., Farhadi, A.: YOLOv3: An Incremental Improvement (2018)

    Google Scholar 

  14. Razakarivony, S., Jurie, F.: Vehicle detection in aerial imagery: a small target detection benchmark. J. Vis. Commun. Image Represent. 34, 187–203 (2015)

    Article  Google Scholar 

  15. Tanner, F., et al.: Overhead imagery research data set - an annotated data library & tools to aid in the development of computer vision algorithms. In: Proceedings - Applied Imagery Pattern Recognition Workshop, pp. 1–8 (2009). https://doi.org/10.1109/AIPR.2009.5466304

  16. Mundhenk, T.N., Konjevod, G., Sakla, W.A., Boakye, K.: A large contextual dataset for classification, detection and counting of cars with deep learning. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 785–800. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_48

    Chapter  Google Scholar 

  17. Xia, G.S., et al.: DOTA: a large-scale dataset for object detection in aerial images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3974–3983 (2018)

    Google Scholar 

  18. Liu, K., Mattyus, G.: Fast multiclass vehicle detection on aerial images. IEEE Geosci. Remote Sens. Lett. 12, 1938–1942 (2015)

    Article  Google Scholar 

  19. Robicquet, A., Sadeghian, A., Alahi, A., Savarese, S.: Learning social etiquette: human trajectory understanding in crowded scenes. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 549–565. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_33

    Chapter  Google Scholar 

  20. Lyu, S., et al.: UA-DETRAC 2018: report of AVSS2018 IWT4S challenge on advanced traffic monitoring. In: Proceedings of the AVSS 2018 - 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (2019)

    Google Scholar 

  21. Hsieh, M.R., Lin, Y.L., Hsu, W.H.: Drone-based object counting by spatially regularized regional proposal network. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4165–4173 (2017)

    Google Scholar 

  22. Zhu, P., Wen, L., Du, D., Bian, X., Ling, H.: VisDrone-VDT2018: the vision meets drone video detection and tracking challenge results, vol. 11206, pp. 1–23 (2018)

    Google Scholar 

  23. OGC (OpenGeoSpatial): Common Database Standard. https://www.opengeospatial.org/standards/cdb

  24. Cheng, P., Zhou, G., Zheng, Z.: Detecting and counting vehicles from small low-cost UAV images. In: American Society for Photogrammetry and Remote Sensing Annual Conference 2009, ASPRS 2009, vol. 1, pp. 138–144 (2009)

    Google Scholar 

  25. Azevedo, C.L., Cardoso, J.L., Ben-Akiva, M., Costeira, J.P., Marques, M.: Automatic vehicle trajectory extraction by aerial remote sensing. Procedia Soc. Behav. Sci. 111, 849–858 (2014)

    Article  Google Scholar 

  26. Zheng, Z., Wang, X., Zhou, G., Jiang, L.: Vehicle detection based on morphology from highway aerial images. In: International Geoscience and Remote Sensing Symposium, pp. 5997–6000 (2012)

    Google Scholar 

  27. Choi, J., Yang, Y.: Vehicle detection from aerial images using local shape information. Adv. Image Video Technol. 5414, 227–236 (2009)

    Article  Google Scholar 

  28. Hinz, S., Stilla, U.: Car detection in aerial thermal images by local and global evidence accumulation. Pattern Recognit. Lett. 27, 308–315 (2006)

    Article  Google Scholar 

  29. Niu, X.: A semi-automatic framework for highway extraction and vehicle detection based on a geometric deformable model. ISPRS J. Photogramm. Remote Sens. 61, 170–186 (2006)

    Article  Google Scholar 

  30. Xu, Y., Yu, G., Wang, Y., Wu, X., Ma, Y.: A hybrid vehicle detection method based on Viola-Jones and HOG + SVM from UAV images. Sensors (Switzerland) 16, 1325 (2016)

    Article  Google Scholar 

  31. Moranduzzo, T., Melgani, F.: Automatic car counting method for unmanned aerial vehicle images. IEEE Trans. Geosci. Remote Sens. 52, 1635–1647 (2014)

    Article  Google Scholar 

  32. Moranduzzo, T., Melgani, F.: Detecting cars in UAV images with a catalog-based approach. IEEE Trans. Geosci. Remote Sens. 52, 6356–6367 (2014)

    Article  Google Scholar 

  33. Sommer, L.W., Schuchert, T., Beyerer, J.: Fast deep vehicle detection in aerial images. In: Proceedings of the 2017 IEEE Winter Conference on Applications of Computer Vision, WACV 2017, pp. 311–319 (2017)

    Google Scholar 

  34. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You Only Look Once: Unified, Real-Time Object Detection (2015)

    Google Scholar 

  35. Westlake, N., Cai, H., Hall, P.: Detecting people in artwork with CNNs. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9913, pp. 825–841. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46604-0_57

    Chapter  Google Scholar 

  36. Zafar, I., Tzanidou, G., Burton, R., Patel, N., Araujo, L.: Hands-On Convolutional Neural Networks with TensorFlow: Solve Computer Vision Problems with Modeling in TensorFlow and Python. Packt Publishing, Birmingham (2018)

    Google Scholar 

  37. Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: CVPR 2017, pp. 7263–7271 (2016)

    Google Scholar 

  38. Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88, 303–338 (2010)

    Article  Google Scholar 

  39. Johnson-Roberson, M., Barto, C., Mehta, R., Sridhar, S.N., Vasudevan, R.: Driving in the matrix: can virtual worlds replace human-generated annotations for real world tasks? In: IEEE International Conference on Robotics and Automation (ICRA), pp. 746–753 (2017)

    Google Scholar 

  40. Gaidon, A., Wang, Q., Cabon, Y., Vig, E.: Virtual worlds as proxy for multi-object tracking analysis. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 4340–4349 (2016)

    Google Scholar 

  41. PPG Industries, Inc.: 2018 Global Color Trend Popularity. https://news.ppg.com/automotive-color-trends. Accessed 01 Aug 2019

  42. Salton, G., McGill, M.: Introduction to Modern Information Retrieval. McGraw-Hill, Inc., New York (1986)

    MATH  Google Scholar 

  43. Carrillo, J., Davis, J., Osorio, J., Goodin, C., Durst, J.: High-fidelity physics-based modeling and simulation for training and testing convolutional neural networks for UGV systems (in review). In: Modelling and Simulation for Autonomous Systems, MESAS 2019 (2019)

    Google Scholar 

  44. Russakovsky, O., Li, L.-J., Fei-Fei, L.: Best of both worlds: human-machine collaboration for object annotation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2121–2131. IEEE (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Flight Systems, Bundeswehr University Munich, Neubiberg, Germany

    Michael Krump, Martin Ruß & Peter Stütz

Authors
  1. Michael Krump
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martin Ruß
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peter Stütz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Krump .

Editor information

Editors and Affiliations

  1. NATO Modelling and Simulation Centre of Excellence, Rome, Italy

    Jan Mazal

  2. MIRPA Lab, University of Palermo, Palermo, Italy

    Adriano Fagiolini

  3. Faculty of Mechanical Engineering, Brno University of Technology, Brno, Czech Republic

    Petr Vasik

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Krump, M., Ruß, M., Stütz, P. (2020). Deep Learning Algorithms for Vehicle Detection on UAV Platforms: First Investigations on the Effects of Synthetic Training. In: Mazal, J., Fagiolini, A., Vasik, P. (eds) Modelling and Simulation for Autonomous Systems. MESAS 2019. Lecture Notes in Computer Science(), vol 11995. Springer, Cham. https://doi.org/10.1007/978-3-030-43890-6_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-43890-6_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-43889-0

  • Online ISBN: 978-3-030-43890-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics