Skip to main content

Advertisement

Springer Nature Link
Log in

Event-triggered multi-agent credit allocation pursuit-evasion algorithm

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

The reinforcement learning is used to study the problem of multi-agent pursuit-evasion games in this article. The main problem of current reinforcement learning applied to multi-agents is the low learning efficiency of agents. To solve this problem, a credit allocation mechanism is adopted in the Multi-agent Deep Deterministic Policy Gradient frame (hereinafter referred to as the MADDPG), the core idea of which is to enable individuals who contribute more to the group to occupy a higher degree of dominance in subsequent training iterations. An event-triggered mechanism is utilized for the simplification of calculation. An observer is set for the feedback value, and the credit allocation algorithm is activated only when the observer believes that the agent group is in a local optimal training dilemma. The final simulation and experiment show that, In most cases, the event-triggered multiagent credit allocation algorithm (hereinafter referred to as the EDMCA algorithm) obtained better results and discussed the parameter settings of the observer.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

References

  1. Mnih V, Kavukcuoglu K, Silver D et al. (2015) Human-level control through deep reinforcement learning. Nature 518:529–533.

    Article  Google Scholar 

  2. Ferber J, Weiss G (1999) Multi-agent systems: an introduction to distributed artificial intelligence. Reading, Addison-Wesley.

    Google Scholar 

  3. Mordatch I, Abbeel P (2017) Emergence of grounded compositional language in multi-agent populations. arXiv:1703.04908

  4. Singh S, Cohn D (1998) How to dynamically merge markov decision processes. Adv Neural Inf Process Syst 10:1057–1063.

    Google Scholar 

  5. Tan M (1993) Multi-agent reinforcement learning: Independent vs. cooperative agents. Proceedings of the tenth international conference on machine learning 330–337

  6. Dayan P, Hinton GE (1993) Feudal reinforcement learning. Advances in neural information processing systems 271–278

  7. Vinyals O, Babuschkin I, Czarnecki WM et al. (2019) Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature 575:350–354.

    Article  Google Scholar 

  8. Littman ML (1994) Markov games as a framework for multi-agent reinforcement learning. Machine Learning Proceedings 1994:157–163.

    Google Scholar 

  9. Sprague N, Ballard D (2003) Multiple-goal reinforcement learning with modular sarsa(0)

  10. Tesauro G (2004) Extending Q-learning to general adaptive multi-agent systems. Advances in neural information processing systems 16:871–878.

    Google Scholar 

  11. Lowe R, Wu Y, Tamar A, Harb J, Abbeel P, Mordatch I (2017) Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in neural information processing systems 30:6379–6390.

    Google Scholar 

  12. Foerster JN, Farquhar G, Afouras T, Nardelli N, Whiteson S (2018) Counterfactual multi-agent policy gradients. Proceedings of the AAAI Conference on Artificial Intelligence 32:2974–2982.

    Article  Google Scholar 

  13. Wang YW, Lei Y, Bian T, Guan ZH (2019) Distributed control of nonlinear multiagent systems with unknown and nonidentical control directions via event-triggered communication. IEEE Transactions on Cybernetics 50:1820–1832.

    Article  Google Scholar 

  14. Guan ZH, Hu B, Chi M, He DX, Cheng XM (2014) Guaranteed performance consensus in second-order multi-agent systems with hybrid impulsive control. Automatica 50:2415–2418.

    Article  MathSciNet  MATH  Google Scholar 

  15. Guan ZH, Hill DJ, Shen X (2005) On hybrid impulsive and switching systems and application to nonlinear control. IEEE Trans Autom Control 50:1058–1062.

    Article  MathSciNet  MATH  Google Scholar 

  16. Foerster J, Nardelli N, Farquhar G, Afouras T, Torr P, Kohli P, Whiteson S (2017) Stabilising experience replay for deep multi-agent reinforcement learning. Proceedings of the 34th International Conference on Machine Learning 70:1146–1155

  17. Bansal T, Pachocki J, Sidor S, Sutskever I, Mordatch I (2017) Emergent complexity via multi-agent competition. arXiv:1710.03748

  18. Yang YD, Luo R, Li M, Zhou M, Zhang WN, Wang J (2018) Mean field multi-agent reinforcement learning. Proceedings of the 35th International Conference on Machine Learning 80:5571–5580

  19. Omidshafiei S, Pazis J, Amato C, How JP, Vian J (2017) Deep decentralized multi-task multi-agent reinforcement learning under partial observability. Proceedings of the 34th International Conference on Machine Learning 70:2681–2690

  20. Gupta J K, Egorov M, Kochenderfer M (2017) Cooperative multi-agent control using deep reinforcement learning. International Conference on Autonomous Agents and Multiagent Systems 66–83

  21. Foerster JN, Assael YM, Freitas N, Whiteson S (2016) Learning to communicate with deep multi-agent reinforcement learning. arXiv:1605.06676

  22. Ghosh A, Kulharia V, Namboodiri VP, Torr P, Dokania PK (2018) Multi-agent diverse generative adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8513–8521

  23. Ahilan S, Dayan P (2019) Feudal multi-agent hierarchies for cooperative reinforcement learning. arXiv:1901.08492

  24. Iqbal S, Sha F (2019) Actor-attention-critic for multi-agent reinforcement learning. International Conference on Machine Learning 2961–2970

  25. Liu K, Duan P, Duan Z, Cai H, Lü J (2018) Leader-following consensus of multi-agent systems with switching networks and event-triggered control. IEEE Transactions on Circuits and Systems-I: Regular Papers 65:1696–1706.

    Article  Google Scholar 

  26. Wu ZG, Xu Y, Pan YJ, Su H, Tang Y (2018) Event-triggered control for consensus problem in multi-agent systems with quantized relative state measurements and external disturbance. IEEE Transactions on Circuits and Systems-I: Regular Papers 65:2232–2242.

    Article  MathSciNet  MATH  Google Scholar 

  27. Luo S, Ye D (2019) Adaptive double event-triggered control for linear multi-agent systems with actuator faults. IEEE Transactions on Circuits and Systems-I: Regular Papers 66:4829–4839.

    Article  MathSciNet  MATH  Google Scholar 

  28. Yang X, Zhu Q (2021) Stabilization of stochastic retarded systems based on sampled-data feedback control. IEEE Transactions on Systems, Man, and Cybernetics: Systems 51:5895–5904.

    Article  Google Scholar 

  29. Zhu Q (2019) Stabilization of stochastic nonlinear delay systems with exogenous disturbances and the event-triggered feedback control. IEEE Trans Autom Control 64:3764–3771.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, 430074, People’s Republic of China

    Bo-Kun Zhang, Bin Hu & Zhi-Hong Guan

  2. School of Petroleum Engineering College, Yangtze University, Jingzhou, 434023, People’s Republic of China

    Ding-Xue Zhang

  3. School of Automation, Central South University, Changsha, 430083, People’s Republic of China

    Xin-Ming Cheng

Authors
  1. Bo-Kun Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Bin Hu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Ding-Xue Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Zhi-Hong Guan
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Xin-Ming Cheng
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Zhi-Hong Guan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

B. Hu, Z.-H. Guan and X.-M. Cheng: This work was partially supported by the National Natural Science Foundation of China under Grants 61976100, 61976099, 62233007, and 61873287.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, BK., Hu, B., Zhang, DX. et al. Event-triggered multi-agent credit allocation pursuit-evasion algorithm. Neural Process Lett 55, 789–802 (2023). https://doi.org/10.1007/s11063-022-10909-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-022-10909-3

Keywords