Abstract
In recent years, deep learning as a subfield of machine learning has gained increasing attention due to its potential advantages in empowering autonomous systems with the ability to automatically learn underlying features in data at different levels of abstractions, to build complex concepts out of simpler ones and to get better with experience without being explicitly programmed. This book chapter provides a comprehensive review on the applications of deep learning in unmanned autonomous vehicles. We focus on particular research efforts that employ deep learning techniques to endow autonomous vehicles with different cognitive functionality, following the cognitive cycle of autonomous vehicles. This cognitive cycle of Sense-Aware-Decide-Act-Adapt-Learn extends the deliberative cycle of Sense-Decide-Act by adding situation awareness, adaptation and learning capabilities to autonomous vehicles. Potential applications of deep learning and major challenges are highlighted in this chapter.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Zafarifar B,Weda H et al (2008) Horizon detection based on sky-color and edge features. In: Visual communications and image processing, vol 6822. International Society for Optics and Photonics, p 682220
Zhan W, Xiao C, Wen Y, Zhou C, Yuan H, Xiu S, Zhang Y, Zou X, Liu X, Li Q (2019) Autonomous visual perception for unmanned surface vehicle navigation in an unknown environment. Sensors 19(10):2216
Rebetez J, Satizábal HF, Mota M, Noll D, Büchi L, Wendling M, Cannelle, B, Pérez-Uribe A, Burgos S (2016) Augmenting a convolutional neural network with local histograms-a case study in crop classification from high-resolution UAV imagery. ESANN
Hui X, Bian J, Zhao X, Tan M (2018) Deep-learning-based autonomous navigation approach for uav transmission line inspection. In: 10th international conference on advanced computational intelligence (ICACI). IEEE, pp 455–460
Khamis A (2019) Biological versus non-biological/artificial intelligence. In: Towards data science
Gopnik A (2017) An ai that knows the world like children do. Scientific Amer. Mind 28(5):21–28
Mitchell T (1997) Machine learning. McGraw-Hill
Halevy A, Norvig P, Pereira F (2009) The unreasonable effectiveness of data. IEEE Intelligent Systems 24(2):8–12
Goodfellow I, Bengio Y, Courville A, Bengio Y (2016) Deep learning, vol 1. MIT Press, Cambridge
Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533
Hinton GE, Osindero S, Teh Y-W (2006) A fast learning algorithm for deep belief nets. Neural computation 18(7):1527–1554
Meyer D (2015) Introduction to autoencoders
Carreira-Perpinan MA, HintonGE (2005) On contrastive divergence learning. In: Aistats, vol 10. Citeseer, pp 33–40
Michigan (2019) Michigan self driving car dataset. Accessed 11 Oct 2020
Oxford (2014) Oxford robot car car dataset. Accessed 11 Oct 2020
Stanford (2009) Stanford self driving car dataset. Accessed 11 Oct 2020
Udacity (2016) Udacity self driving car dataset. Accessed 11 Oct 2020
Pitropov M, GarciaD, Rebello J, Smart M, Wang C, Czarnecki K, Waslander S (2020) Canadian adverse driving conditions dataset.arXiv preprint arXiv:2001.10117
Waymo (2019) Waymo self driving car dataset. Accessed 11 Oct 2020
Scape A (2018) Apollo Scape self driving car dataset. Accessed 11 Oct 2020
Geiger A, Lenz P, Stiller C, Urtasun R (2013) Vision meets robotics: the kitti dataset. Int J Robot Res (IJRR)
Khamis A (2021) Smart mobility: exploring foundational technologies and wider impacts. APress (Springer Nature), ISBN: 978-1-4842-7101-8
Endsley MR (2016) Designing for situation awareness: an approach to user-centered design. CRC Press
Council NR et al (2012) NASA space technology roadmaps and priorities: restoring NASA’s technological edge and paving the way for a new era in space. National Academies Press
Draper V (1994) Environmental restoration and waste management program teleoperator hand controllers: contextual human factors assessment. OAK Ridge National Laboratory, Departamento de Energia de los Estados Unidos, Reporte
Bayat B, Bermejo-Alonso J, Carbonera J, Facchinetti T, Fiorini S, Goncalves P, Jorge VA, Habib M, Khamis A, Melo K et al (2016) Requirements for building an ontology for autonomous robots. Ind Robot Int J 43(5)
Hollnagel E (2009) The four cornerstones of resilience engineering
International S (2016) Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles. SAE Int (J3016)
Jung S, Hwang S, Shin H, Shim DH (2018) Perception, guidance, and navigation for indoor autonomous drone racing using deep learning. IEEE Robotics and Automation Letters 3(3):2539–2544
Kim D, Ryu H, Yonchorhor J, Shim DH (2020) A deep-learning-aided automatic vision-based control approach for autonomous drone racing in game of drones competition. In: NeurIPS competition and demonstration track. PMLR, pp 37–46
Mammeri A, Boukerche A, Tang Z (2016) A real-time lane marking localization, tracking and communication system. Computer Communications 73:132–143
Niu J, Lu J, Xu M, Lv P, Zhao X (2016) Robust lane detection using two-stage feature extraction with curve fitting. Pattern Recognition 59:225–233
Yi S-C, Chen Y-C, Chang C-H (2015) A lane detection approach based on intelligent vision. Computers & Electrical Engineering 42:23–29
Perng JW, Hsu YW, Yang YZ, Chen CY, Yin TK (2020) Development of an embedded road boundary detection system based on deep learning. In: Image and vision computing, p 103935
Liu B, Liu H, Yuan J (2019) Lane line detection based on mask R-CNN. In: 3rd international conference on mechatronics engineering and information technology (ICMEIT). Atlantis Press
Wang Z, Ren W, Qiu Q (2018) Lanenet: real-time lane detection networks for autonomous driving. arXiv preprint arXiv:1807.01726
Kim J, Kim J, Jang G-J, Lee M (2017) Fast learning method for convolutional neural networks using extreme learning machine and its application to lane detection. Neural Networks 87:109–121
Chen Z, Liu Q, Lian C (2019) Pointlanenet: efficient end-to-end CNNS for accurate real-time lane detection. In: IEEE intelligent vehicles symposium (IV). IEEE, pp 2563–2568
Ma Y, Havyarimana V, Bai J, Xiao Z (2018) Vision-based lane detection and lane-marking model inference: a three-step deep learning approach. In: 9th international symposium on parallel architectures, algorithms and programming (PAAP). IEEE, pp 183–190
Lee S, Xie K, Ngoduy D, Keyvan-Ekbatani M (2019) An advanced deep learning approach to real-time estimation of lane-based queue lengths at a signalized junction. Transportation research part C: emerging technologies 109:117–136
Zakaria N, Shapiai M, Rahman MA, Yahya W (2020) Lane line detection via deep learning based-approach applying two types of input into network model. J Soc Automot Eng Malaysia 4(2)
Nassar A, Amer K, ElHakim R, ElHelw M (2018) A deep CNN-based framework for enhanced aerial imagery registration with applications to UAV geolocalization. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 1513–1523
Bejiga MB, Zeggada A, Nouffidj A, Melgani F (2017) A convolutional neural network approach for assisting avalanche search and rescue operations with uav imagery. Remote Sensing 9(2):100
Liu T, Fu HY, Wen Q, Zhang DK, Li LF (2018) Extended faster R-CNN for long distance human detection: Finding pedestrians in UAV images. In: IEEE international conference on consumer electronics (ICCE). IEEE, pp 1–2
Cherian A, Andersh J, Morellas V, Papanikolopoulos N, Mettler B (2009)Autonomous altitude estimation of a UAV using a single onboard camera. In: IEEE/RSJ international conference on intelligent robots and systems. IEEE, pp 3900–3905
Barták R, Vomlelová M (2017) Using machine learning to identify activities of a flying drone from sensor readings. In: 13th international flairs conference
Delmerico J, Giusti A, Mueggler E, Gambardella LM, Scaramuzza D (2016) On-the-spot training for terrain classification in autonomous air-ground collaborative teams. In: International symposium on experimental robotics Springer, pp 574–585
Qu C, Gai W, Zhong M, Zhang J (2020) A novel reinforcement learning based grey wolf optimizer algorithm for unmanned aerial vehicles (UAVS) path planning. Appl Soft Comput 8
Zhang B, Liu W, Mao Z, Liu J, Shen L (2014) Cooperative and geometric learning algorithm (cgla) for path planning of uavs with limited information. Automatica 50(3):809–820
Junell JL, Van Kampen EJ, de Visser CC, Chu QP (2015) Reinforcement learning applied to a quadrotor guidance law in autonomous flight. In: AIAA guidance, navigation, and control conference, p 1990
Loquercio A, Maqueda AI, Del-Blanco CR, Scaramuzza D (2018) Dronet: Learning to fly by driving. IEEE Robotics and Automation Letters 3(2):1088–1095
Yang S, Konam S, Ma C, Rosenthal S, Veloso M, Scherer S (2017) Obstacle avoidance through deep networks based intermediate perception. arXiv preprint arXiv:1704.08759
Schultz AC, Grefenstette JJ (2000) Continuous and embedded learning in autonomous vehicles: Adapting to sensor failures. In: Unmanned ground vehicle technology II, vol 4024. International Society for Optics and Photonics, pp 55–62
Kira Z, Schultz AC (2006) Continuous and embedded learning for multi-agent systems. In: IEEE/RSJ international conference on intelligent robots and systems. IEEE, pp 3184–3190
Ahlawat A, Kabir KS, Pathak K, Singh S, Kaushal S, Kumar S Smart surveillance using on cloud machine learning and internet controlled UAVS
Kellenberger B, Marcos D, Lobry S, Tuia D (2019) Half a percent of labels is enough: Efficient animal detection in uav imagery using deep cnns and active learning. IEEE Transactions on Geoscience and Remote Sensing 57(12):9524–9533
Panico A, Fragonara LZ, Al-Rubaye S (2020) Adaptive detection tracking system for autonomous uav maritime patrolling. In: IEEE 7th international workshop on metrology for aerospace (MetroAeroSpace), pp 539–544
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Khamis, A., Patel, D., Elgazzar, K. (2021). Deep Learning for Unmanned Autonomous Vehicles: A Comprehensive Review. In: Koubaa, A., Azar, A.T. (eds) Deep Learning for Unmanned Systems. Studies in Computational Intelligence, vol 984. Springer, Cham. https://doi.org/10.1007/978-3-030-77939-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-77939-9_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-77938-2
Online ISBN: 978-3-030-77939-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)