Skip to main content
Springer Nature Link
Log in

On Autonomous Drone Navigation Using Deep Learning and an Intelligent Rainbow DQN Agent

  • Conference paper
  • First Online:
Intelligent Data Engineering and Automated Learning – IDEAL 2022 (IDEAL 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13756))

  • 1567 Accesses

Abstract

Drones are intelligent devices that offer solutions for a continuously expanding variety of applications. Therefore, there would be a significant improvement if these systems could explore space automatically and without human-supervision. This work integrates cutting-edge artificial intelligence techniques that allow drones to travel independently. Following an overview of reinforcement learning methods built for discrete action space settings, a multilayer Perceptron model is constructed for feature extraction along with Hybrid neural networks. The agent employed in the experiments is a Rainbow DQN agent trained on the AirSim simulator. The experimental results are encouraging as the agent was tested for 16 missions and the accuracy was higher than 93%. In particular, the success for action selection was 97% and 93 for mission success. Finally, future work related to the navigation of autonomous drones is discussed including current concepts and methods of integration with more sophisticated algorithmic approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    The agent is not aware of the map, i.e. the layout of the obstacles. However, he knows the direction where the target is located. Therefore, it can estimate the direction to shift it with a larger one pace to target.

  2. 2.

    https://microsoft.github.io/AirSim/.

  3. 3.

    https://github.com/segment_tree.py.

References

  1. Karras, A., Karras, C., Giotopoulos, K.C., Giannoukou, I., Tsolis, D., Sioutas, S.: Download speed optimization in P2P networks using decision making and adaptive learning. In: Daimi, K., Al Sadoon, A. (eds.) Proceedings of the ICR’22 International Conference on Innovations in Computing Research. ICR 2022. Advances in Intelligent Systems and Computing, vol. 1431, pp. 225–238. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-14054-9_22

  2. Amer, K., Samy, M., Shaker, M., ElHelw, M.: Deep convolutional neural network based autonomous drone navigation. In: Thirteenth International Conference on Machine Vision, vol. 11605, pp. 16–24. SPIE (2021)

    Google Scholar 

  3. De Asis, K., Hernandez-Garcia, J., Holland, G., Sutton, R.: Multi-step reinforcement learning: a unifying algorithm. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)

    Google Scholar 

  4. Hessel, M., et al.: Rainbow: combining improvements in deep reinforcement learning. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)

    Google Scholar 

  5. Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)

    Google Scholar 

  6. Roghair, J., Niaraki, A., Ko, K., Jannesari, A.: A vision based deep reinforcement learning algorithm for UAV obstacle avoidance. In: Arai, K. (ed.) IntelliSys 2021. LNNS, vol. 294, pp. 115–128. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-82193-7_8

    Chapter  Google Scholar 

  7. Siciliano, B., Khatib, O., Kröger, T.: Springer Handbook of Robotics, vol. 200. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-30301-5

    Book  MATH  Google Scholar 

  8. Um, J.S.: Drones as Cyber-Physical Systems. Springer, Singapore (2019). https://doi.org/10.1007/978-981-13-3741-3

    Book  Google Scholar 

  9. Van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double q-learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Engineering and Informatics Department, University of Patras, Patras, Greece

    Andreas Karatzas, Aristeidis Karras, Christos Karras & Spyros Sioutas

  2. Department of Management Science and Technology, University of Patras, Patras, Greece

    Konstantinos C. Giotopoulos

  3. Department of Informatics, Ionian University, Corfu, Greece

    Konstantinos Oikonomou

Authors
  1. Andreas Karatzas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aristeidis Karras
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christos Karras
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Konstantinos C. Giotopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Konstantinos Oikonomou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Spyros Sioutas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christos Karras .

Editor information

Editors and Affiliations

  1. University of Manchester, Manchester, UK

    Hujun Yin

  2. Technical University of Madrid, Madrid, Spain

    David Camacho

  3. University of Birmingham, Birmingham, UK

    Peter Tino

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Karatzas, A., Karras, A., Karras, C., Giotopoulos, K.C., Oikonomou, K., Sioutas, S. (2022). On Autonomous Drone Navigation Using Deep Learning and an Intelligent Rainbow DQN Agent. In: Yin, H., Camacho, D., Tino, P. (eds) Intelligent Data Engineering and Automated Learning – IDEAL 2022. IDEAL 2022. Lecture Notes in Computer Science, vol 13756. Springer, Cham. https://doi.org/10.1007/978-3-031-21753-1_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-21753-1_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-21752-4

  • Online ISBN: 978-3-031-21753-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics