Skip to main content
SpringerLink
Log in

Deep Learning for Unmanned Autonomous Vehicles: A Comprehensive Review

  • Chapter
  • First Online:
Deep Learning for Unmanned Systems

Part of the book series: Studies in Computational Intelligence ((SCI,volume 984))

Abstract

In recent years, deep learning as a subfield of machine learning has gained increasing attention due to its potential advantages in empowering autonomous systems with the ability to automatically learn underlying features in data at different levels of abstractions, to build complex concepts out of simpler ones and to get better with experience without being explicitly programmed. This book chapter provides a comprehensive review on the applications of deep learning in unmanned autonomous vehicles. We focus on particular research efforts that employ deep learning techniques to endow autonomous vehicles with different cognitive functionality, following the cognitive cycle of autonomous vehicles. This cognitive cycle of Sense-Aware-Decide-Act-Adapt-Learn extends the deliberative cycle of Sense-Decide-Act by adding situation awareness, adaptation and learning capabilities to autonomous vehicles. Potential applications of deep learning and major challenges are highlighted in this chapter.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Zafarifar B,Weda H et al (2008) Horizon detection based on sky-color and edge features. In: Visual communications and image processing, vol 6822. International Society for Optics and Photonics, p 682220

    Google Scholar 

  2. Zhan W, Xiao C, Wen Y, Zhou C, Yuan H, Xiu S, Zhang Y, Zou X, Liu X, Li Q (2019) Autonomous visual perception for unmanned surface vehicle navigation in an unknown environment. Sensors 19(10):2216

    Article  Google Scholar 

  3. Rebetez J, Satizábal HF, Mota M, Noll D, Büchi L, Wendling M, Cannelle, B, Pérez-Uribe A, Burgos S (2016) Augmenting a convolutional neural network with local histograms-a case study in crop classification from high-resolution UAV imagery. ESANN

    Google Scholar 

  4. Hui X, Bian J, Zhao X, Tan M (2018) Deep-learning-based autonomous navigation approach for uav transmission line inspection. In: 10th international conference on advanced computational intelligence (ICACI). IEEE, pp 455–460

    Google Scholar 

  5. Khamis A (2019) Biological versus non-biological/artificial intelligence. In: Towards data science

    Google Scholar 

  6. Gopnik A (2017) An ai that knows the world like children do. Scientific Amer. Mind 28(5):21–28

    Article  Google Scholar 

  7. Mitchell T (1997) Machine learning. McGraw-Hill

    Google Scholar 

  8. Halevy A, Norvig P, Pereira F (2009) The unreasonable effectiveness of data. IEEE Intelligent Systems 24(2):8–12

    Article  Google Scholar 

  9. Goodfellow I, Bengio Y, Courville A, Bengio Y (2016) Deep learning, vol 1. MIT Press, Cambridge

    Google Scholar 

  10. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533

    Google Scholar 

  11. Hinton GE, Osindero S, Teh Y-W (2006) A fast learning algorithm for deep belief nets. Neural computation 18(7):1527–1554

    Article  MathSciNet  Google Scholar 

  12. Meyer D (2015) Introduction to autoencoders

    Google Scholar 

  13. Carreira-Perpinan MA, HintonGE (2005) On contrastive divergence learning. In: Aistats, vol 10. Citeseer, pp 33–40

    Google Scholar 

  14. Michigan (2019) Michigan self driving car dataset. Accessed 11 Oct 2020

    Google Scholar 

  15. Oxford (2014) Oxford robot car car dataset. Accessed 11 Oct 2020

    Google Scholar 

  16. Stanford (2009) Stanford self driving car dataset. Accessed 11 Oct 2020

    Google Scholar 

  17. Udacity (2016) Udacity self driving car dataset. Accessed 11 Oct 2020

    Google Scholar 

  18. Pitropov M, GarciaD, Rebello J, Smart M, Wang C, Czarnecki K, Waslander S (2020) Canadian adverse driving conditions dataset.arXiv preprint arXiv:2001.10117

  19. Waymo (2019) Waymo self driving car dataset. Accessed 11 Oct 2020

    Google Scholar 

  20. Scape A (2018) Apollo Scape self driving car dataset. Accessed 11 Oct 2020

    Google Scholar 

  21. Geiger A, Lenz P, Stiller C, Urtasun R (2013) Vision meets robotics: the kitti dataset. Int J Robot Res (IJRR)

    Google Scholar 

  22. Khamis A (2021) Smart mobility: exploring foundational technologies and wider impacts. APress (Springer Nature), ISBN: 978-1-4842-7101-8

    Google Scholar 

  23. Endsley MR (2016) Designing for situation awareness: an approach to user-centered design. CRC Press

    Google Scholar 

  24. Council NR et al (2012) NASA space technology roadmaps and priorities: restoring NASA’s technological edge and paving the way for a new era in space. National Academies Press

    Google Scholar 

  25. Draper V (1994) Environmental restoration and waste management program teleoperator hand controllers: contextual human factors assessment. OAK Ridge National Laboratory, Departamento de Energia de los Estados Unidos, Reporte

    Google Scholar 

  26. Bayat B, Bermejo-Alonso J, Carbonera J, Facchinetti T, Fiorini S, Goncalves P, Jorge VA, Habib M, Khamis A, Melo K et al (2016) Requirements for building an ontology for autonomous robots. Ind Robot Int J 43(5)

    Google Scholar 

  27. Hollnagel E (2009) The four cornerstones of resilience engineering

    Google Scholar 

  28. International S (2016) Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles. SAE Int (J3016)

    Google Scholar 

  29. Jung S, Hwang S, Shin H, Shim DH (2018) Perception, guidance, and navigation for indoor autonomous drone racing using deep learning. IEEE Robotics and Automation Letters 3(3):2539–2544

    Article  Google Scholar 

  30. Kim D, Ryu H, Yonchorhor J, Shim DH (2020) A deep-learning-aided automatic vision-based control approach for autonomous drone racing in game of drones competition. In: NeurIPS competition and demonstration track. PMLR, pp 37–46

    Google Scholar 

  31. Mammeri A, Boukerche A, Tang Z (2016) A real-time lane marking localization, tracking and communication system. Computer Communications 73:132–143

    Article  Google Scholar 

  32. Niu J, Lu J, Xu M, Lv P, Zhao X (2016) Robust lane detection using two-stage feature extraction with curve fitting. Pattern Recognition 59:225–233

    Article  Google Scholar 

  33. Yi S-C, Chen Y-C, Chang C-H (2015) A lane detection approach based on intelligent vision. Computers & Electrical Engineering 42:23–29

    Article  Google Scholar 

  34. Perng JW, Hsu YW, Yang YZ, Chen CY, Yin TK (2020) Development of an embedded road boundary detection system based on deep learning. In: Image and vision computing, p 103935

    Google Scholar 

  35. Liu B, Liu H, Yuan J (2019) Lane line detection based on mask R-CNN. In: 3rd international conference on mechatronics engineering and information technology (ICMEIT). Atlantis Press

    Google Scholar 

  36. Wang Z, Ren W, Qiu Q (2018) Lanenet: real-time lane detection networks for autonomous driving. arXiv preprint arXiv:1807.01726

  37. Kim J, Kim J, Jang G-J, Lee M (2017) Fast learning method for convolutional neural networks using extreme learning machine and its application to lane detection. Neural Networks 87:109–121

    Article  Google Scholar 

  38. Chen Z, Liu Q, Lian C (2019) Pointlanenet: efficient end-to-end CNNS for accurate real-time lane detection. In: IEEE intelligent vehicles symposium (IV). IEEE, pp 2563–2568

    Google Scholar 

  39. Ma Y, Havyarimana V, Bai J, Xiao Z (2018) Vision-based lane detection and lane-marking model inference: a three-step deep learning approach. In: 9th international symposium on parallel architectures, algorithms and programming (PAAP). IEEE, pp 183–190

    Google Scholar 

  40. Lee S, Xie K, Ngoduy D, Keyvan-Ekbatani M (2019) An advanced deep learning approach to real-time estimation of lane-based queue lengths at a signalized junction. Transportation research part C: emerging technologies 109:117–136

    Article  Google Scholar 

  41. Zakaria N, Shapiai M, Rahman MA, Yahya W (2020) Lane line detection via deep learning based-approach applying two types of input into network model. J Soc Automot Eng Malaysia 4(2)

    Google Scholar 

  42. Nassar A, Amer K, ElHakim R, ElHelw M (2018) A deep CNN-based framework for enhanced aerial imagery registration with applications to UAV geolocalization. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 1513–1523

    Google Scholar 

  43. Bejiga MB, Zeggada A, Nouffidj A, Melgani F (2017) A convolutional neural network approach for assisting avalanche search and rescue operations with uav imagery. Remote Sensing 9(2):100

    Article  Google Scholar 

  44. Liu T, Fu HY, Wen Q, Zhang DK, Li LF (2018) Extended faster R-CNN for long distance human detection: Finding pedestrians in UAV images. In: IEEE international conference on consumer electronics (ICCE). IEEE, pp 1–2

    Google Scholar 

  45. Cherian A, Andersh J, Morellas V, Papanikolopoulos N, Mettler B (2009)Autonomous altitude estimation of a UAV using a single onboard camera. In: IEEE/RSJ international conference on intelligent robots and systems. IEEE, pp 3900–3905

    Google Scholar 

  46. Barták R, Vomlelová M (2017) Using machine learning to identify activities of a flying drone from sensor readings. In: 13th international flairs conference

    Google Scholar 

  47. Delmerico J, Giusti A, Mueggler E, Gambardella LM, Scaramuzza D (2016) On-the-spot training for terrain classification in autonomous air-ground collaborative teams. In: International symposium on experimental robotics Springer, pp 574–585

    Google Scholar 

  48. Qu C, Gai W, Zhong M, Zhang J (2020) A novel reinforcement learning based grey wolf optimizer algorithm for unmanned aerial vehicles (UAVS) path planning. Appl Soft Comput 8

    Google Scholar 

  49. Zhang B, Liu W, Mao Z, Liu J, Shen L (2014) Cooperative and geometric learning algorithm (cgla) for path planning of uavs with limited information. Automatica 50(3):809–820

    Article  MathSciNet  Google Scholar 

  50. Junell JL, Van Kampen EJ, de Visser CC, Chu QP (2015) Reinforcement learning applied to a quadrotor guidance law in autonomous flight. In: AIAA guidance, navigation, and control conference, p 1990

    Google Scholar 

  51. Loquercio A, Maqueda AI, Del-Blanco CR, Scaramuzza D (2018) Dronet: Learning to fly by driving. IEEE Robotics and Automation Letters 3(2):1088–1095

    Article  Google Scholar 

  52. Yang S, Konam S, Ma C, Rosenthal S, Veloso M, Scherer S (2017) Obstacle avoidance through deep networks based intermediate perception. arXiv preprint arXiv:1704.08759

  53. Schultz AC, Grefenstette JJ (2000) Continuous and embedded learning in autonomous vehicles: Adapting to sensor failures. In: Unmanned ground vehicle technology II, vol 4024. International Society for Optics and Photonics, pp 55–62

    Google Scholar 

  54. Kira Z, Schultz AC (2006) Continuous and embedded learning for multi-agent systems. In: IEEE/RSJ international conference on intelligent robots and systems. IEEE, pp 3184–3190

    Google Scholar 

  55. Ahlawat A, Kabir KS, Pathak K, Singh S, Kaushal S, Kumar S Smart surveillance using on cloud machine learning and internet controlled UAVS

    Google Scholar 

  56. Kellenberger B, Marcos D, Lobry S, Tuia D (2019) Half a percent of labels is enough: Efficient animal detection in uav imagery using deep cnns and active learning. IEEE Transactions on Geoscience and Remote Sensing 57(12):9524–9533

    Article  Google Scholar 

  57. Panico A, Fragonara LZ, Al-Rubaye S (2020) Adaptive detection tracking system for autonomous uav maritime patrolling. In: IEEE 7th international workshop on metrology for aerospace (MetroAeroSpace), pp 539–544

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. General Motors Canada, Oshawa, Canada

    Alaa Khamis

  2. Ontario Tech University, Oshawa, Canada

    Dipkumar Patel & Khalid Elgazzar

Authors
  1. Alaa Khamis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dipkumar Patel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Khalid Elgazzar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alaa Khamis .

Editor information

Editors and Affiliations

  1. College of Computer and Information Sciences, Prince Sultan University, Riyadh, Saudi Arabia

    Anis Koubaa

  2. College of Computer and Information Sciences, Prince Sultan University, Riyadh, Saudi Arabia

    Ahmad Taher Azar

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Khamis, A., Patel, D., Elgazzar, K. (2021). Deep Learning for Unmanned Autonomous Vehicles: A Comprehensive Review. In: Koubaa, A., Azar, A.T. (eds) Deep Learning for Unmanned Systems. Studies in Computational Intelligence, vol 984. Springer, Cham. https://doi.org/10.1007/978-3-030-77939-9_1

Download citation

Publish with us

Policies and ethics

Search

Navigation