Skip to main content
SpringerLink
Log in

Roadway Detection Using Convolutional Neural Network Through Camera and LiDAR Data

  • Conference paper
  • First Online:
Hybrid Artificial Intelligent Systems (HAIS 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13469))

Included in the following conference series:

  • 826 Accesses

Abstract

Roadway detection is one of the main topics that autonomous vehicles must face to safety navigate along roads. In this paper, we present the architecture and results of a roadway detection system which uses both camera and LiDAR data to segment the road surface from a Bird’s-eye view. Discussion about how camera and LiDAR data has been combined is presented along with example images to later discuss about the neural model that has been developed. The proposed method performs among other state-of-the-art methods on the Kitti-Road dataset. Finally, future research lines are introduced, and it is discussed how the use of the full LiDAR FOV could bring benefits for road detection.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Alvarez, J., Lopez, A., Baldrich, R.: Illuminant-invariant model-based road segmentation, pp. 1175–1180. IEEE (2008). https://doi.org/10.1109/IVS.2008.4621283

  2. Bogue, R.: The role of artificial intelligence in robotics. Ind. Robot: Int. J. 41, 119–123 (2014). https://doi.org/10.1108/IR-01-2014-0300

    Article  Google Scholar 

  3. Bolte, J.A., Bar, A., Lipinski, D., Fingscheidt, T.: Towards corner case detection for autonomous driving, pp. 438–445 (2019). https://doi.org/10.1109/IVS.2019.8813817. ISSN 2642-7214

  4. Caltagirone, L., Bellone, M., Svensson, L., Wahde, M.: Lidar-camera fusion for road detection using fully convolutional neural networks. Robot. Auton. Syst. 111, 125–131 (2019). https://doi.org/10.1016/J.ROBOT.2018.11.002

    Article  Google Scholar 

  5. Caltagirone, L., Scheidegger, S., Svensson, L., Wahde, M.: Fast lidar-based road detection using fully convolutional neural networks, pp. 1019–1024. IEEE (2017). https://doi.org/10.1109/IVS.2017.7995848

  6. Chen, L., Yang, J., Kong, H.: Lidar-histogram for fast road and obstacle detection. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 1343–1348 (2017). https://doi.org/10.1109/ICRA.2017.7989159

  7. Chen, L., Chen, P., Lin, Z.: Artificial intelligence in education: a review. IEEE Access 8, 75264–75278 (2020). https://doi.org/10.1109/ACCESS.2020.2988510

    Article  Google Scholar 

  8. Chen, Z., Tao, D., Zhang, J.: Progressive lidar adaptation for road detection. IEEE/CAA J. Automatica Sinica 6, 693–702 (2019)

    Article  Google Scholar 

  9. van Dyk, D.A., Meng, X.L.: The art of data augmentation. J. Comput. Graph. Stat. 10, 1–50 (2001). https://doi.org/10.1198/10618600152418584

    Article  MathSciNet  Google Scholar 

  10. Fritsch, J., Kühnl, T., Geiger, A.: A new performance measure and evaluation benchmark for road detection algorithms. In: 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013) (2013)

    Google Scholar 

  11. Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., Garcia-Rodriguez, J.: A review on deep learning techniques applied to semantic segmentation. arXiv preprint arXiv:1704.06857 (2017)

  12. Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. 32, 1231–1237 (2013). https://doi.org/10.1177/0278364913491297

    Article  Google Scholar 

  13. González, A., et al.: Pedestrian detection at day/night time with visible and fir cameras: a comparison. Sensors 16, 820 (2016). https://doi.org/10.3390/s16060820

    Article  Google Scholar 

  14. Gu, S., Yang, J., Kong, H.: A cascaded lidar-camera fusion network for road detection, pp. 13308–13314. IEEE (2021). https://doi.org/10.1109/ICRA48506.2021.9561935

  15. Gu, S., Zhang, Y., Tang, J., Yang, J., Kong, H.: Road detection through CRF based lidar-camera fusion. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 3832–3838 (2019). https://doi.org/10.1109/ICRA.2019.8793585

  16. Gu, S., Zhang, Y., Yang, J., Alvarez, J.M., Kong, H.: Two-view fusion based convolutional neural network for urban road detection. In: IEEE International Conference on Intelligent Robots and Systems, pp. 6144–6149 (2019). https://doi.org/10.1109/IROS40897.2019.8968054

  17. Gu, S., Zhang, Y., Yuan, X., Yang, J., Wu, T., Kong, H.: Histograms of the normalized inverse depth and line scanning for urban road detection. IEEE Trans. Intell. Transp. Syst. 20, 3070–3080 (2019). https://doi.org/10.1109/TITS.2018.2871945

    Article  Google Scholar 

  18. Hamet, P., Tremblay, J.: Artificial intelligence in medicine. Metabolism 69, S36–S40 (2017). https://doi.org/10.1016/j.metabol.2017.01.011

    Article  Google Scholar 

  19. Han, X., Wang, H., Lu, J., Zhao, C.: Road detection based on the fusion of lidar and image data. Int. J. Adv. Robotic Syst. 14, 172988141773810 (2017). https://doi.org/10.1177/1729881417738102

  20. Hu, X., Rodriguez, F.S.A., Gepperth, A.: A multi-modal system for road detection and segmentation, pp. 1365–1370. IEEE (2014). https://doi.org/10.1109/IVS.2014.6856466

  21. Jung, C.R., Kelber, C.R.: Lane following and lane departure using a linear-parabolic model. Image Vis. Comput. 23, 1192–1202 (2005). https://doi.org/10.1016/j.imavis.2005.07.018

    Article  Google Scholar 

  22. Kaur, G., Kumar, D.: Lane detection techniques: a review. Int. J. Comput. Appl. (2015)

    Google Scholar 

  23. Kröger, F.: Automated driving in its social, historical and cultural contexts. In: Maurer, M., Gerdes, J.C., Lenz, B., Winner, H. (eds.) Autonomous Driving, pp. 41–68. Springer, Heidelberg (2016). https://doi.org/10.1007/978-3-662-48847-8_3

    Chapter  Google Scholar 

  24. Li, Y., Ibanez-Guzman, J.: Lidar for autonomous driving: the principles, challenges, and trends for automotive lidar and perception systems. IEEE Signal Process. Mag. 37, 50–61 (2020). https://doi.org/10.1109/MSP.2020.2973615. Conference Name: IEEE Signal Processing Magazine

  25. Lyu, Y., Bai, L., Huang, X.: ChipNet: real-time lidar processing for drivable region segmentation on an FPGA. IEEE Trans. Circuits Syst. I: Regular Papers 66, 1769–1779 (2019). https://doi.org/10.1109/TCSI.2018.2881162

    Article  Google Scholar 

  26. Magazine, T.: Science: Radio auto. Time Magazine (1925)

    Google Scholar 

  27. Martínez-Díaz, M., Soriguera, F., Pérez, I.: Autonomous driving: a bird’s eye view. IET Intell. Transp. Syst. 13, 563–579 (2019). https://doi.org/10.1049/iet-its.2018.5061

    Article  Google Scholar 

  28. Mokhtarzade, M., Zoej, M.V.: Road detection from high-resolution satellite images using artificial neural networks. Int. J. Appl. Earth Obs. Geoinf. 9, 32–40 (2007). https://doi.org/10.1016/j.jag.2006.05.001

    Article  Google Scholar 

  29. Nayar, S., Narasimhan, S.: Vision in bad weather. In: Proceedings of the Seventh IEEE International Conference on Computer Vision (1999)

    Google Scholar 

  30. Ogden, K.: The effects of paved shoulders on accidents on rural highways. Accid. Anal. Prev. 29, 353–362 (1997). https://doi.org/10.1016/S0001-4575(97)00001-8

    Article  Google Scholar 

  31. Rashed, H., Ramzy, M., Vaquero, V., Sallab, A.E., Sistu, G., Yogamani, S.: FuseMODNet: real-time camera and lidar based moving object detection for robust low-light autonomous driving. In: Proceedings of the 2019 International Conference on Computer Vision Workshop, ICCVW 2019, pp. 2393–2402 (2019). https://doi.org/10.1109/ICCVW.2019.00293

  32. Raviteja, S., Shanmughasundaram, R.: Advanced driver assistance system (ADAS), pp. 737–740. IEEE (2018). https://doi.org/10.1109/ICCONS.2018.8663146

  33. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  34. Sakaridis, C., Dai, D., Gool, L.V.: Semantic foggy scene understanding with synthetic data. Int. J. Comput. Vis. 126, 973–992 (2018)

    Article  Google Scholar 

  35. Shima, T., Nagasaki, T., Kuriyama, A., Yoshimura, K., Sobue, T.: Fundamental technologies driving the evolution of autonomous driving. Hitachi Rev. 65, 427 (2016)

    Google Scholar 

  36. Shinzato, P.Y., Wolf, D.F., Stiller, C.: Road terrain detection: avoiding common obstacle detection assumptions using sensor fusion. In: Proceedings of the IEEE Intelligent Vehicles Symposium, pp. 687–692 (2014). https://doi.org/10.1109/IVS.2014.6856454

  37. Shinzato, P.Y.: Estimation of obstacles and road area with sparse 3D points. Universidade de São Paulo (2015). https://doi.org/10.11606/T.55.2015.tde-07082015-100709

  38. Shorten, C., Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. J. Big Data 6, 60 (2019). https://doi.org/10.1186/s40537-019-0197-0

    Article  Google Scholar 

  39. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2015)

    Google Scholar 

  40. Wong, K., Gu, Y., Kamijo, S.: Mapping for autonomous driving: opportunities and challenges. IEEE Intell. Transp. Syst. Mag. 13, 91–106 (2021). https://doi.org/10.1109/MITS.2020.3014152. Conference Name: IEEE Intelligent Transportation Systems Magazine

  41. Xiao, L., Dai, B., Liu, D., Hu, T., Wu, T.: CRF based road detection with multi-sensor fusion. In: Proceedings of the IEEE Intelligent Vehicles Symposium, pp. 192–198 (2015). https://doi.org/10.1109/IVS.2015.7225685

  42. Xiao, L., Dai, B., Liu, D., Zhao, D., Wu, T.: Monocular road detection using structured random forest. Int. J. Adv. Robot. Syst. 13, 101 (2016). https://doi.org/10.5772/63561

    Article  Google Scholar 

  43. Xiao, L., Wang, R., Dai, B., Fang, Y., Liu, D., Wu, T.: Hybrid conditional random field based camera-lidar fusion for road detection. Inf. Sci. 432, 543–558 (2018). https://doi.org/10.1016/J.INS.2017.04.048

    Article  MathSciNet  Google Scholar 

  44. Yang, F., Yang, J., Jin, Z., Wang, H.: A fusion model for road detection based on deep learning and fully connected CRF. In: 2018 13th System of Systems Engineering Conference, SoSE 2018, pp. 29–36 (2018). https://doi.org/10.1109/SYSOSE.2018.8428696

  45. Yenikaya, S., Yenikaya, G., Düven, E.: Keeping the vehicle on the road - a survey on on-road lane detection systems. ACM Comput. Surv. 46, 1–43 (2013). https://doi.org/10.1145/2522968.2522970

    Article  Google Scholar 

  46. Zaletelj, J., Burnik, U., Tasic, J.F.: Registration of satellite images based on road network map, pp. 49–53. IEEE (2013). https://doi.org/10.1109/ISPA.2013.6703713

Download references

Author information

Authors and Affiliations

  1. SECOMUCI Research Group, Escuela de Ingenierías Industrial e Informática, Universidad de León, Campus de Vegazana s/n, 24071, León, Spain

    Martín Bayón-Gutiérrez, Jose Aveleira-Mata, Héctor Alaiz-Moretón & María Teresa García-Ordás

  2. SALBIS Research Group, Department of Electric, Systems and Automatics Engineering, Universidad de León, Campus of Vegazana s/n, León, 24071, León, Spain

    José Alberto Benítez-Andrades & Sergio Rubio-Martín

Authors
  1. Martín Bayón-Gutiérrez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. José Alberto Benítez-Andrades
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sergio Rubio-Martín
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jose Aveleira-Mata
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Héctor Alaiz-Moretón
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. María Teresa García-Ordás
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martín Bayón-Gutiérrez .

Editor information

Editors and Affiliations

  1. University of Deusto, Bilbao, Spain

    Pablo García Bringas

  2. University of León, León, Spain

    Hilde Pérez García

  3. University of La Rioja, Logroño, La Rioja, Spain

    Francisco Javier Martínez de Pisón

  4. University of Oviedo, Oviedo, Spain

    José Ramón Villar Flecha

  5. Data Science and Big Data Analytics Lab, Pablo de Olavide University, Sevilla, Spain

    Alicia Troncoso Lora

  6. Department of Computer Science, University of Oviedo, Oviedo, Spain

    Enrique A. de la Cal

  7. Applied Computational Intelligence, University of Burgos, Burgos, Burgos, Spain

    Álvaro Herrero

  8. Universidad Pablo de Olavide, Seville, Spain

    Francisco Martínez Álvarez

  9. DIGIP, University of Bergamo, Dalmine, Bergamo, Italy

    Giuseppe Psaila

  10. Department of Industrial Engineering, University of A Coruña, Ferrol, Spain

    Héctor Quintián

  11. University of Salamanca, Salamanca, Spain

    Emilio Corchado

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bayón-Gutiérrez, M., Benítez-Andrades, J.A., Rubio-Martín, S., Aveleira-Mata, J., Alaiz-Moretón, H., García-Ordás, M.T. (2022). Roadway Detection Using Convolutional Neural Network Through Camera and LiDAR Data. In: García Bringas, P., et al. Hybrid Artificial Intelligent Systems. HAIS 2022. Lecture Notes in Computer Science(), vol 13469. Springer, Cham. https://doi.org/10.1007/978-3-031-15471-3_36

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-15471-3_36

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-15470-6

  • Online ISBN: 978-3-031-15471-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation