Skip to main content
SpringerLink
Log in

Towards LiDAR and RADAR Fusion for Object Detection and Multi-object Tracking in CARLA Simulator

  • Conference paper
  • First Online:
ROBOT2022: Fifth Iberian Robotics Conference (ROBOT 2022)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 590))

Included in the following conference series:

  • 813 Accesses

Abstract

Detection and Multi-Object Tracking (DAMOT) systems have a critical role to play in scene understanding in the context of autonomous driving. Modern Autonomous Driving Stacks (ADS) require a software processing unit or module that allows them to understand the data in the environment and convert it into vital information for further decision making. In this context, this work develops a DAMOT module based on Machine Learning techniques, such as DBSCAN or BEV-SORT, that receives information from LiDAR and RADAR sensors in CARLA Simulator. This module uses containerisation techniques with Docker and standard robotics communications with ROS. The performance of the method is evaluated in terms of detection in the AD PerDevKit dataset, developed by the authors.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Gog, I., Kalra, S., Schafhalter, P., Wright, M.A., Gonzalez, J.E., Stoica, I.: Pylot: a modular platform for exploring latency-accuracy tradeoffs in autonomous vehicles. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 8806–8813. IEEE (2021)

    Google Scholar 

  2. Chekroun, R., Toromanoff, M., Hornauer, S., Moutarde, F.: Gri: general reinforced imitation and its application to vision-based autonomous driving. arXiv preprint. arXiv:2111.08575 (2021)

  3. Gómez-Huélamo, C., et al.: How to build and validate a safe and reliable autonomous driving stack? a ros based software modular architecture baseline. In: 2022 IEEE Intelligent Vehicles Symposium (IV). IEEE (2022)

    Google Scholar 

  4. Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V.: CARLA: an open urban driving simulator. In: Levine, S., Vanhoucke, V., Goldberg, K. (ed.) Proceedings of the 1st Annual Conference on Robot Learning, vol. 78 of Proceedings of Machine Learning Research, 13–15 Nov 2017, pp. 1–16. PMLR

    Google Scholar 

  5. Patole, S.M., Torlak, M., Wang, D., Ali, M.: Automotive radars: a review of signal processing techniques. IEEE Sig. Process. Mag. 34(2), 22–35 (2017)

    Article  Google Scholar 

  6. de la Pena, J., Bergasa, L. M., Antunes, M., Arango, F., Gómez-Huélamo, C., López-Guillén, E.: Ad perdevkit: an autonomous driving perception development kit using carla simulator and ros. In: 2022 IEEE International Conference on Intelligent Transportation Systems (ITSC). IEEE (2022)

    Google Scholar 

  7. Zhangjing Wang, Y.W., Niu, Q.: Multi-sensor fusion in automated driving: a survey. IEEE Access 8, 2847–2868 (2019)

    Article  Google Scholar 

  8. Qi, C. R., Yi, L., Su, H., Guibas, L.J.: Pointnet++: deep hierarchical feature learning on point sets in a metric space. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  9. Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., Beijbom, O.: Pointpillars: fast encoders for object detection from point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12697–12705 (2019)

    Google Scholar 

  10. Scheiner, N., Kraus, F., Appenrodt, N., Dickmann, J., Sick, B.: Object detection for automotive radar point clouds-a comparison. AI Perspect. 3(1), 1–23 (2021)

    Article  Google Scholar 

  11. Ester, M., Kriegel, H.-P., Sander, J., Xiaowei, X., et al.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: KDD, vol. 96, pp. 226–231 (1996)

    Google Scholar 

  12. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  13. Redmon, J., Farhadi, A.: Yolov3: an incremental improvement. arXiv preprint. arXiv:1804.02767 (2018)

  14. Cho, H., Seo, Y.W., Kumar, B.V., Rajkumar, R.R.: A multi-sensor fusion system for moving object detection and tracking in urban driving environments. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 1836–1843. IEEE (2014)

    Google Scholar 

  15. Julier, S.J., Uhlmann, J.K.: New extension of the kalman filter to nonlinear systems. In: Signal processing, sensor fusion, and target recognition VI, vol. 3068, pp. 182–193. International Society for Optics and Photonics (1997)

    Google Scholar 

  16. Gómez-Huélamo, C., Bergasa, L.M., Gutiérrez, R., Arango, J.F., Díaz, A.: Smartmot: exploiting the fusion of hdmaps and multi-object tracking for real-time scene understanding in intelligent vehicles applications. In: 2021 IEEE Intelligent Vehicles Symposium (IV), pp. 710–715. IEEE (2021)

    Google Scholar 

  17. Bewley, A., Ge, Z., Ott, L., Ramos, F., Upcroft, B.: Simple online and realtime tracking. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 3464–3468. IEEE (2016)

    Google Scholar 

  18. Yurtsever, E., Lambert, J., Carballo, A., Takeda, K.: A survey of autonomous driving: common practices and emerging technologies. IEEE Access 8, 58443–58469 (2020)

    Article  Google Scholar 

  19. Merkel, D., et al.: Docker: lightweight linux containers for consistent development and deployment. Linux J. 2014(239), 2 (2014)

    Google Scholar 

  20. Quigley, M., et al. Ros: an open-source robot operating system. In: ICRA Workshop on Open Source Software, Kobe, Japan, vol. 3, p. 5 (2009)

    Google Scholar 

  21. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. In: Communications of the ACM, vol. 24, no. 6, pp. 381–395 (1981)

    Google Scholar 

  22. Schubert, E., Sander, J., Ester, M., Kriegel, H.P., Xu, X.: Dbscan revisited, revisited: why and how you should (still) use DBSCAN. ACM Trans. Database Syst. (TODS) 42(3), 1–21 (2017)

    Article  MathSciNet  Google Scholar 

  23. Schumann, O., et al.: A real-world radar point cloud data set for automotive applications. In: RadarScenes (2021)

    Google Scholar 

  24. Kalman, R.E.: A new approach to linear filtering and prediction problems. J. Basic Eng. 82(1), 35–45 (1960)

    Article  MathSciNet  Google Scholar 

  25. Kuhn, H.W.: The hungarian method for the assignment problem. Naval Res. Logistics Q. 2(1–2), 83–97 (1955)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work has been funded in part from the Spanish MICINN/FEDER through the Techs4AgeCar project (RTI2018-099263-B-C21) and from the RoboCity2030-DIH-CM project (P2018/NMT-4331), funded by Programas de actividades I+D (CAM), cofunded by EU Structural Funds and Scholarship for Introduction to Research activity by University of Alcalá.

Author information

Authors and Affiliations

  1. Electronics Department, Universidad de Alcalá, Alcalá de Henares, Spain

    Santiago Montiel-Marín, Carlos Gómez-Huélamo, Javier de la Peña, Miguel Antunes, Elena López-Guillén & Luis M. Bergasa

Authors
  1. Santiago Montiel-Marín
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlos Gómez-Huélamo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Javier de la Peña
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Miguel Antunes
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Elena López-Guillén
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Luis M. Bergasa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Santiago Montiel-Marín .

Editor information

Editors and Affiliations

  1. Centro Universitario de la Defensa (CUD), Zaragoza, Spain

    Danilo Tardioli

  2. Grupo de Robótica, Universidad de León, León, Spain

    Vicente Matellán

  3. GRVC Robotics Lab, Escuela Técnica Superior de Ingeniería, Universidad de Sevilla, Sevilla, Spain

    Guillermo Heredia

  4. School of Engineering, Polytechnic Institute of Porto, Porto, Portugal

    Manuel F. Silva

  5. Institute of Systems and Robotics, University of Coimbra, Coimbra, Portugal

    Lino Marques

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Montiel-Marín, S., Gómez-Huélamo, C., de la Peña, J., Antunes, M., López-Guillén, E., Bergasa, L.M. (2023). Towards LiDAR and RADAR Fusion for Object Detection and Multi-object Tracking in CARLA Simulator. In: Tardioli, D., Matellán, V., Heredia, G., Silva, M.F., Marques, L. (eds) ROBOT2022: Fifth Iberian Robotics Conference. ROBOT 2022. Lecture Notes in Networks and Systems, vol 590. Springer, Cham. https://doi.org/10.1007/978-3-031-21062-4_45

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-21062-4_45

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-21061-7

  • Online ISBN: 978-3-031-21062-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation