Skip to main content
SpringerLink
Log in

Dual deep Q-learning network guiding a multiagent path planning approach for virtual fire emergency scenarios

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

With continuous deterioration of the natural environment and the corresponding significant increase in the occurrence of disasters, forest fire accidents have frequently occurred in recent decades. Therefore, it is important to perform extensive effective fire drills to increase evacuation experience and emergency reaction capacity. In comparison to traditional fire drills, which are subject to many latent uncertainties and incur high costs, fire exercises based on virtual scenarios offer many advantages, such as low cost and high safety. Accordingly, the planning and design of effective evacuation paths that sufficiently match real conditions have become an imperative focus of related research. In this paper, we propose a novel framework for path planning in virtual emergency scenarios, which consists of three parts. (a) Configuration of the virtual environment: for convenience in handling, the virtual emergency scenario is discretized into many individual grid cells. (b) Policy generation: a dual deep Q-learning network approach is employed to obtain an effective policy that can allow agents to intelligently find effective paths. (c) Grouping strategy: a strategy is proposed to support multiple agents in achieving collective evacuation based on a given policy. Finally, extensive experiments are presented to validate the superiority of the proposed framework. The results show that by comparison with the existing related state-of-the-art methods, our proposed framework is superior and feasible.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Van Hasselt H, Guez A, Silver D (2016) Deep reinforcement learning with Double Q-learning[C]. Proceedings of the AAAI Conference on Artificial Intelligence 30(1)

  2. Haznedar B, Arslan MT, Kalinli A (2021) Optimizing ANFIS using simulated annealing algorithm for classification of microarray gene expression cancer data[J]. Medical & Biological Engineering & Computing 59(3):497–509

    Article  Google Scholar 

  3. Wang B, Xie Y, Zhou S et al (2017) Reversible Data Hiding Based on DNA Computing[J]. Computational Intelligenceand Neuroscience 1-9

  4. Adleman L (1994) Molecular computation of solutions to combinatorial problems[J]. Science 266(5187):1021–1024

    Article  Google Scholar 

  5. Zhong G, Li T, Jiao W et al (2020) DNA computing inspired deep networks design[J]. Neurocomputing 3(24):140–147

    Article  Google Scholar 

  6. Hussien HH (2019) DNA computing for RGB image encryption with genetic algorithm[C]. In 2019 14th international conference on computer engineering and systems (ICCES). IEEE, 169-173

  7. Sun L, Kong X, Xu J et al (2019) A hybrid gene selection method based on ReliefF and ant colony optimization algorithm for tumor classification[J]. Sci Rep 9(1):1–14

    Google Scholar 

  8. Mondal M, Ray KS (2020) Prediction of visibility under radiation fog by DNA computing[J]. New Mathematics and Natural Computation 16(2):231–254

    Article  Google Scholar 

  9. Jafarzadeh N, Iranmanesh A (2016) A new graph theoretical method for analyzing DNA sequences based on genetic codes[J]. MATCH-Commun Math Comput Chem 75(3):731–742

    MathSciNet  MATH  Google Scholar 

  10. Liu R, Wang Y (2019) Research on TSP Solution Based on Genetic Algorithm[C]. In 2019 IEEE/ACIS 18th International Conference on Computer and Information Science (ICIS). IEEE

  11. Yang R, Zhang C, Gao R (2017) A new bionic method inspired by DNA computation to solve the hamiltonian path problem[C]. 2017 IEEE International Conference on Information and Automation (ICIA). IEEE 219-225

  12. Elsayed WM, Elmogy M, El-Desouky BS (2021) DNA sequence reconstruction based on innovated hybridization technique of probabilistic cellular automata and particle swarm optimization. Inf Sci 1(547):828–840

    Article  MathSciNet  MATH  Google Scholar 

  13. Li X, Wang B, Lv H et al (2020) Constraining DNA sequences with a triplet-bases unpaired. IEEE Trans Nanobiosci 19(2):299–303

    Article  Google Scholar 

  14. Shi K, Huang L, Jiang D, Sun Y, Tong X, Xie Y, Fang Z (2022) Path planning optimization of intelligent vehicle based on improved genetic and ant colony hybrid algorithm[J]. Frontiers in Bioengineering and Biotechnology 10:905–983

    Article  Google Scholar 

  15. Liu Y, Jiang D, Xu C et al (2022) Deep learning based 3D target detection for indoor scenes[J]. Applied Intelligence 1-14

  16. Yun J, Jiang D, Sun Y, Huang L, Tao B, Jiang G, Kong J, Weng Y, Li G, Fang Z (2022) Grasping pose detection for loose stacked object based on convolutional neural network with multiple self-powered sensors information[J]. IEEE Sensors Journal

  17. Liu H, Xu B, Lu D et al (2018) A path planning approach for crowd evacuation in buildings based on improved artificial bee colony algorithm[J]. Appl Soft Comput 68:360–376

    Article  Google Scholar 

  18. Da Silva FL, Costa AHR (2019) A survey on transfer learning for multiagent reinforcement learning systems[J]. Journal of Artificial Intelligence Research 64:645–703

    Article  MathSciNet  MATH  Google Scholar 

  19. Gronauer S, Diepold K (2022) Multi-agent deep reinforcement learning: A survey[J]. Artif Intell Rev 55(2):895–943

    Article  Google Scholar 

  20. Wang X, Wang S, Liang X et al (2022) Deep reinforcement learning: a survey[J]. IEEE Transactions on Neural Networks and Learning Systems 1-15

  21. Oroojlooy A, Hajinezhad D (2022) A review of cooperative multi-agent deep reinforcement learning[J]. Applied Intelligence 1-46

  22. Yin Z, Yang J, Zhang Q et al (2021) DNA computing model for satisfiability problem based on hybridization chain reaction[J]. Int J Pattern Recognit Artif Intell 35(03):2159–2170

    Article  Google Scholar 

  23. Qiu C, Hu Y, Chen Y et al (2019) Deep deterministic policy gradient (DDPG)-based energy harvesting wireless communications[J]. IEEE Internet Things J 6(5):8577–8588

    Article  Google Scholar 

  24. Prasad PC, Jaiswal A, Shakya S et al (2021) Portfolio Optimization: A Study of Nepal Stock Exchange[C]. Proceedings of International Conference on Sustainable Expert Systems. Springer, Singapore 659-672

  25. Li J, Chen Y, Zhao XN et al (2022) An improved DQN path planning algorithm[J]. J Supercomput 78(1):616–639

    Article  MathSciNet  Google Scholar 

  26. Zuo G, Du T, Lu J (2017) Double DQN method for object detection[C]. 2017 Chinese Automation Congress (CAC). IEEE 6727-6732

  27. Min K, Kim H, Huh K. (2018) Deep Q Learning based high level driving policy determination[C]. IEEE Intelligent Vehicles Symposium (IV). IEEE 226-231

  28. Mnih V, Kavukcuoglu K, Silver D et al (2015) Human-level control through deep reinforcement learning[J]. Nature 518(7540):529–533

  29. Duan Y, Chen X, Houthooft R, Schulman J, Abbeel P (2016) Benchmarking deep reinforcement learning for continuous control[C]. In International conference on machine learning (PMLR) 1329-1338

  30. Zhou W, Jiang W, Jie B et al (2022) Multiagent evacuation framework for a virtual fire emergency scenario based on generative adversarial imitation learning[J]. Computer Animation and Virtual Worlds 33(1):e2035

    Article  Google Scholar 

  31. Li J, Chen Y, Zhao XN et al (2022) An improved DQN path planning algorithm[J]. J Supercomput 78(1):616–639

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors appreciate the comments and suggestions from all the anonymous reviewers, which have helped to significantly improve this paper. In addition, this work was supported in part by the National Natural Science Foundation of China (NSFC) (grant no. 61902003) and the Doctoral Scientific Research Foundation of Anhui Normal University.

Funding

Partial financial support was received from the National Natural Science Foundation of China (NSFC) (grant no. 61902003).

Author information

Authors and Affiliations

  1. School of Computer and Information, Anhui Normal University, Wuhu, 241002, China

    Wen Zhou, Chen Zhang & Siyuan Chen

Authors
  1. Wen Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Siyuan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wen Zhou.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, W., Zhang, C. & Chen, S. Dual deep Q-learning network guiding a multiagent path planning approach for virtual fire emergency scenarios. Appl Intell 53, 21858–21874 (2023). https://doi.org/10.1007/s10489-023-04601-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04601-9

Keywords

Search

Navigation