Skip to main content
SpringerLink
Log in

Sensor Fusion Approach for an Autonomous Shunting Locomotive

  • Conference paper
  • First Online:
Informatics in Control, Automation and Robotics (ICINCO 2017)

Part of the book series: Lecture Notes in Electrical Engineering ((LNEE,volume 495))

Abstract

In order to allow robust obstacle detection for autonomous freight traffic using freight trains or shunting locomotives, several different sensors are required. Humans and other objects must be detected so that the vehicle can stop in time. Laser scanners deliver distance information and are popular in robotics and automation. Cameras deliver further pieces of information on the environment and are especially useful for the classification of objects, but do not deliver distance measurements. Thermal cameras are ideal for the detection of humans based on their body temperature if the surrounding temperature is not too similar. It is only the combination of these different sensors which delivers enough robustness. Therefore a sensor fusion and an extrinsic calibration has to take place. This article presents an approach fusing a 2D and an 8-layer 3D laser scanner with a thermal and a Red-Green-Blue (RGB) camera, using a triangular calibration target taking all six degrees of freedom into account. The calibration was tested and the results validated during reference measurements and autonomous and manually controlled field tests. This sensor fusion approach was used for the obstacle detection of an autonomous shunting locomotive.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Autonom fahrende Züge: Deutsche Bahn testet Loks ohne Führer. http://www.wiwo.de/ unternehmen/dienstleister/autonom-fahrende-zuege-deutsche-bahn-testet-loks-ohne-fuehrer/12542272.html

  2. Autonome LKWs sind zu schnell für den Menschen. http://www.sueddeutsche.de/auto/verkehr-autonome-lkws-sind-zu-schnell-fuer-den-menschen-1.3530699

  3. Rail Freight Masterplan. http://www.bmvi.de/SharedDocs/EN/publications/rail-freight-masterplan.pdf?__blob=publicationFile

  4. shz.de. https://www.shz.de/deutschland-welt/wirtschaft/ erster-toter-teslas-autopilot-erkennt-weissen-lkw-nicht-id14150641. html

  5. Daimler. https://www.daimler.com/de/Digitalisierung/automatisierung.html

  6. Gebauer, O., Pree, P., Stadlmann, B.: Autonomously driving trains on open tracks: Concepts, system architecture and implementation aspects. Inf. Technol. 54(6), 266–279 (2012)

    Article  Google Scholar 

  7. Bosch Kollisionswarnsystem für Stadt- und Straßenbahn. http://www.auto.de/magazin/ kollisionswarner-fuer-stadt-und-strassenbahnen/

  8. DE \(|\) Bosch Kollisionswarnsystem für Stadt- und Straßenbahnen. https://www.youtube.com/ watch?v=UG637CAsoAI

  9. Gschwandter, M.: Support Framework for Obstacle Detection on Autonomous Trains. Ph.D. thesis, Universität Salzburg

    Google Scholar 

  10. Unnikrishnan, R., Hebert, M.: Fast Extrinsic Calibration of a Laser Rangefinder to a Camera (2005)

    Google Scholar 

  11. Meierhold, N., Spehr, M., Schilling, A., Gumhold, S., Maas, HG: Automatic feature matching between digital images and 2D representations of a 3D laser scanner point cloud. In: The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. 38, pp. 446–451 (2010)

    Google Scholar 

  12. Alba, M. I., Barazzetti, L., Scaioni, M., Rosina, E., Previtali, M.: Mapping infrared data on terrestrial laser scanning 3D models of buildings. Remote Sens. 3(9) (2011). http://www.mdpi.com/2072-4292/3/9/1847

    Article  Google Scholar 

  13. Borrmann, D., Elseberg, J., Nüchter, A.: Thermal 3D mapping of building Façades. In: Intelligent Autonomous Systems 12: Volume 1 Proceedings of the 12th International Conference IAS-12, pp. 173–182. Jeju Island, Korea, held June 26–29, (2012)

    Google Scholar 

  14. Gong, X., Lin, Y., Liu, J.: 3D LIDAR-camera extrinsic calibration using an arbitrary trihedron. Sensors 13, 1902 (2013). http://www.mdpi.com/1424-8220/13/2/1902

    Article  Google Scholar 

  15. Pandey, G., McBride, J., Savarese, S., Eustice, R.: Extrinsic calibration of a 3D laser scanner and an omnidirectional camera. In: IFAC Proceedings Volumes vol. 43, pp. 226–341. https://doi.org/10.3182/20100906-3-IT-2019.00059. http://www.sciencedirect.com/science/article/pii/S1474667016350790

    Article  Google Scholar 

  16. Liebelt, T.-M.: Fusion von 3D-Laserscanner-Daten mit 2DThermal- Bilddaten. Bachelorthesis, Westfälische Hochschule Gelsenkirchen (2013)

    Google Scholar 

  17. SmokeBot Deliverable 3.2: Software Toolkit Indoor Surface Temperature Mapping. http://aass.oru.se/Research/mro/smokebot/deliverables/SmokeBot_D3_2.pdf

  18. Debattisti, S., Mazzei, L., Panciroli, M.: Automated extrinsic laser and camera inter-calibration using triangular targets. In: 2013 IEEE Intelligent Vehicles Symposium (IV), pp. 696–701 (2013). https://doi.org/10.1109/IVS.2013.6629548

  19. Florez, R., Alberto, S., Fremont, V.: Extrinsic calibration between a multi-layer lidar and a camera. In: IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, pp. 214–219. MFI (2008)

    Google Scholar 

  20. Rodriguez-Garavito, C.H., Ponz, A., Garcia, F., Martin, D., Escalera, A., Amingol, J.M.: Automatic laser and camera extrinsic calibration for data fusion using road plane. In: 17th International Conference on Information Fusion, FUSION 2014, Salamanca, Spain, July 7–10 (2014). http://ieeexplore.ieee.org/document/6916258/

  21. Kämpfchen, N.: Feature-Level Fusion of Laser Scanner and Video Data for Advanced Driver Assistance Systems. Ph.D. thesis, Universität Ulm (2007)

    Google Scholar 

  22. Garcia, F., Olmeda, D., Armingol, J.M., de la Escalera, A.: Hybrid fusion scheme for pedestrian detection based on laser scanner and far infrared camera. In: 2010 IEEE Intelligent Vehicles Symposium (2010)

    Google Scholar 

  23. Wasielewski, S., Strauss, O.: Calibration of a multi-sensor system laser rangefinder/camera. In: Proceedings of the IEEE Xplore Conference: Intelligent Vehicles ’95 Symposium (1995)

    Google Scholar 

  24. Zhang, Q., Pless, R.: Extrinsic calibration of a camera and laser range finder (improves camera calibration). In: Proceedings of 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2004) (2004)

    Google Scholar 

  25. Kassir, A., Peynot, T.: Reliable automatic camera-laser calibration. In: Australasian Conference on Robotics and Automation (2010). https://doi.org/10.1109/IVS.1995.528327

  26. Naroditsky, O., Patterson, A., Daniilidis, K.: Automatic alignment of a camera with a line scan LIDAR system. In: 2011 IEEE International Conference on Robotics and Automation (ICRA), pp. 3429–3434 (2011)

    Google Scholar 

  27. Zeng, L., Ding, M., Zhang, T., Sun, Z.: A fast moving object detection method based on 2D laser scanner and infrared camera. In: Proceedings of SPIE vol. 9675, pp. 96752K–96752K–5

    Google Scholar 

  28. Dong, W., Isler, V.: A novel method for extrinsic calibration of a 2-D laser-rangefinder and a camera. In: CoRR (2016). http://arxiv.org/abs/1603.04132

  29. Operating instructions. https://www.sick.com/media/docs/2/02/802/Operating_instructions_ LD_MRS_3D_LiDAR_sensors_de_IM0032802.PDF

  30. Gleichauf, J., Pfitzner, C., May, S.: Sensor fusion of a 2D laser scanner and a thermal camera. In: Proceedings of the 14th International Conference on Informatics in Control, Automation and Robotics - Vol. 1: ICINCO, pp. 398–405 (2017). ISBN 978-989-758-263-9

    Google Scholar 

  31. May, S.: 3D Time-of-Flight Ranging for Robotic Perception in Dynamic Environments. Dissertation. Universität Osnabrück (2009)

    Google Scholar 

  32. sick\(\_\)ldmrs\(\_\)laser. https://github.com/SICKAG/sick_ldmrs_laser

  33. Optris PI 640. http://www.optris.de/infrarotkamera-optris-pi-640

  34. optris\(\_\)drivers. http://wiki.ros.org/optris_drivers

  35. Groovy ROS node for camera module of Raspberry Pi. https://github.com/fpasteau/ raspicam_node

  36. camera\(\_\)calibration. http://wiki.ros.org/camera_calibration

  37. laser\(\_\)filters. http://wiki.ros.org/laser_filters

  38. solvePnP. http://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#solvepnp

  39. Zhang, Z.: A flexible new technique for camera calibration. In: Proceedings of IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, pp. 1330–1344

    Article  Google Scholar 

  40. Optikkalkulator. http://www.optris.de/optikkalkulator-fuer-waermebildkameras

  41. Gleichauf, J., May, S.: Obstacle avoidance system for an autonomous shunting locomotive. In: Applied Research Conference 2017, Munich, Germany (2017)

    Google Scholar 

  42. Lowe, D.G.: Object recognition from local scale-invariant features. In: Proceedings of the International Conference on Computer Vision (1999)

    Google Scholar 

  43. find\(\_\)object\(\_\)2d. http://wiki.ros.org/find_object_2d

  44. people\(\_\)detect. http://wiki.ros.org/opencv_apps#people_detect

  45. Gleichauf, J.: Sensorfusion für die Hinderniserkennung einer autonomen Rangierlok. Master thesis. Technische Hochschule Nürnberg (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Technische Hochschule Nuremberg, Georg Simon Ohm, Kesslerplatz 12, 90489, Nuremberg, Germany

    Johanna Gleichauf, Johannes Vollet, Christian Pfitzner, Philipp Koch & Stefan May

Authors
  1. Johanna Gleichauf
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Johannes Vollet
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christian Pfitzner
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Philipp Koch
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Stefan May
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Johanna Gleichauf .

Editor information

Editors and Affiliations

  1. Ford Research and Advanced Engineering, Dearborn, MI, USA

    Oleg Gusikhin

  2. University of Paris-EST Créteil (UPEC), Vitry sur Seine, France

    Kurosh Madani

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gleichauf, J., Vollet, J., Pfitzner, C., Koch, P., May, S. (2020). Sensor Fusion Approach for an Autonomous Shunting Locomotive. In: Gusikhin, O., Madani, K. (eds) Informatics in Control, Automation and Robotics . ICINCO 2017. Lecture Notes in Electrical Engineering, vol 495. Springer, Cham. https://doi.org/10.1007/978-3-030-11292-9_30

Download citation

Publish with us

Policies and ethics

Search

Navigation