Skip to main content
SpringerLink
Log in

Performance Evaluation of a DQN-Based Autonomous Aerial Vehicle Mobility Control Method in Corner Environment

  • Conference paper
  • First Online:
Advanced Information Networking and Applications (AINA 2022)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 450))

Abstract

In this paper, we present a DQN-based AAV mobility control method and show the performance evaluation results for a corner environment scenario. The performance evaluation results show that the proposed method can decide the destination based on LiDAR for TLS-DQN. Also, the visualization results of AAV movement show that the TLS-DQN can reach the destination in the corner environment.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 219.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 279.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Stöcker, C., et al.: Review of the current state of UAV regulations. Remote Sens. 9(5), 1–26 (2017)

    Article  Google Scholar 

  2. Artemenko, O., et al.: Energy-aware trajectory planning for the localization of mobile devices using an unmanned aerial vehicle. In: Proceedings of The 25-th International Conference on Computer Communication and Networks (ICCCN 2016), pp. 1-9 (2016)

    Google Scholar 

  3. Popović, M., et al.: An informative path planning framework for UAV-based terrain monitoring. Auton. Robot. 44, 889–911 (2020)

    Article  Google Scholar 

  4. Nguyen, H., et al.: LAVAPilot: lightweight UAV trajectory planner with situational awareness for embedded autonomy to track and locate radio-tags. arXiv:2007.15860, pp. 1–8 (2020)

  5. Oda, T., et al.: Design and implementation of a simulation system based on deep Q-network for mobile actor node control in wireless sensor and actor networks. In: Proceedings of The 31-th IEEE International Conference on Advanced Information Networking and Applications Workshops (IEEE AINA 2017), pp. 195–200 (2017)

    Google Scholar 

  6. Oda, T., et al.: Performance evaluation of a deep Q-network based simulation system for actor node mobility control in wireless sensor and actor networks considering three-dimensional environment. In: Proceedings of The 9-th International Conference on Intelligent Networking and Collaborative Systems (INCoS 2017), pp. 41–52 (2017)

    Google Scholar 

  7. Oda, T., Kulla, E., Katayama, K., Ikeda, M., Barolli, L.: A deep Q-network based simulation system for actor node mobility control in WSANS considering three-dimensional environment: a comparison study for normal and uniform distributions. In: Barolli, L., Javaid, N., Ikeda, M., Takizawa, M. (eds.) CISIS 2018. AISC, vol. 772, pp. 842–852. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-93659-8_77

  8. Sandino, J., et al.: UAV framework for autonomous onboard navigation and people/object detection in cluttered indoor environments. Remote Sens. 12(20), 1–31 (2020)

    Article  Google Scholar 

  9. Moulton, J., et al.: An autonomous surface vehicle for long term operations. In: Proceedings of MTS/IEEE OCEANS, pp. 1–10 (2018)

    Google Scholar 

  10. Oda, T., et al.: Design of a deep Q-network based simulation system for actuation decision in ambient intelligence. In: Proceedings of The 33-rd International Conference on Advanced Information Networking and Applications (AINA 2019), pp. 362–370 (2019)

    Google Scholar 

  11. Oda, T., et al.: Design and implementation of an IoT-based E-learning testbed. Int. J. Web Grid Serv. 13(2), 228–241 (2017)

    Article  Google Scholar 

  12. Hirota, Y., Oda, T., Saito, N., Hirata, A., Hirota, M., Katatama, K.: Proposal and experimental results of an ambient intelligence for training on soldering iron holding. In: Barolli, L., Takizawa, M., Enokido, T., Chen, H.-C., Matsuo, K. (eds.) BWCCA 2020. LNNS, vol. 159, pp. 444–453. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-61108-8_44

  13. Hayosh, D., et al.: Woody: low-cost, open-source humanoid torso robot. In: Proceedings of The 17-th International Conference on Ubiquitous Robots (ICUR 2020), pp. 247–252 (2020)

    Google Scholar 

  14. Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)

    Article  Google Scholar 

  15. Mnih, V., et al.: Playing atari with deep reinforcement learning. arXiv:1312.5602, pp. 1–9 (2013)

  16. Lei, T., Ming, L.: A robot exploration strategy based on Q-learning network. In: IEEE International Conference on Real-time Computing and Robotics (IEEE RCAR 2016), pp. 57–62 (2016)

    Google Scholar 

  17. Riedmiller, M.: Neural fitted Q iteration - first experiences with a data efficient neural reinforcement learning method. In: Proceedings of The 16-th European Conference on Machine Learning (ECML 2005), pp. 317–328 (2005)

    Google Scholar 

  18. Lin, L.J.: Reinforcement learning for robots using neural networks. In: Proceedings of Technical Report, DTIC Document (1993)

    Google Scholar 

  19. Lange, S., Riedmiller, M.: Deep auto-encoder neural networks in reinforcement learning. In: Proceedings of The International Joint Conference on Neural Networks (IJCNN 2010), pp. 1–8 (2010)

    Google Scholar 

  20. Kaelbling, L.P., et al.: Planning and acting in partially observable stochastic domains. Artif. Intell. 101(1–2), 99–134 (1998)

    Article  MathSciNet  Google Scholar 

  21. Saito, N., et al.: A Tabu list strategy based DQN for AAV mobility in indoor single-path environment: implementation and performance evaluation. Internet Things 14, 100394 (2021)

    Article  Google Scholar 

  22. Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of The 13-th International Conference on Artificial Intelligence and Statistics (AISTATS 2010), pp. 249–256 (2010)

    Google Scholar 

  23. Glorot, X., et al.: Deep sparse rectifier neural networks. In: Proceedings of The 14-th International Conference on Artificial Intelligence and Statistics (AISTATS 2011), pp. 315–323 (2011)

    Google Scholar 

  24. Glover, F.: Tabu search - part I. ORSA J. Comput. 1(3), 190–206 (1989)

    Article  Google Scholar 

Download references

Acknowledgement

This work was supported by JSPS KAKENHI Grant Number JP20K19793 and Grant for Promotion of OUS Research Project (OUS-RP-20-3).

Author information

Authors and Affiliations

  1. Graduate School of Engineering, Okayama University of Science (OUS), 1-1 Ridaicho, Kita-ku, Okayama, 700-0005, Japan

    Nobuki Saito & Aoto Hirata

  2. Department of Information and Computer Engineering, Okayama University of Science (OUS), 1-1 Ridaicho, Kita-ku, Okayama, 700-0005, Japan

    Tetsuya Oda, Chihiro Yukawa, Kyohei Toyoshima & Tomoaki Matsui

  3. Department of Information and Communication Engineering, Fukuoka Institute of Technology, 3-30-1 Wajiro-higashi, Higashi-ku, Fukuoka, 811-0295, Japan

    Leonard Barolli

Authors
  1. Nobuki Saito
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tetsuya Oda
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aoto Hirata
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chihiro Yukawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kyohei Toyoshima
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tomoaki Matsui
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Leonard Barolli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tetsuya Oda .

Editor information

Editors and Affiliations

  1. Department of Information and Communication Engineering, Fukuoka Institute of Technology, Fukuoka, Japan

    Leonard Barolli

  2. University of Technology Sydney, Sydney, NSW, Australia

    Farookh Hussain

  3. Faculty of Bussiness Administration, Rissho University, Tokyo, Japan

    Tomoya Enokido

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Saito, N. et al. (2022). Performance Evaluation of a DQN-Based Autonomous Aerial Vehicle Mobility Control Method in Corner Environment. In: Barolli, L., Hussain, F., Enokido, T. (eds) Advanced Information Networking and Applications. AINA 2022. Lecture Notes in Networks and Systems, vol 450. Springer, Cham. https://doi.org/10.1007/978-3-030-99587-4_31

Download citation

Publish with us

Policies and ethics

Search

Navigation