Skip to main content
SpringerLink
Log in

Fuzzy adaptive dynamic programming-based optimal leader-following consensus for heterogeneous nonlinear multi-agent systems

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In this paper, a novel online iterative scheme, based on fuzzy adaptive dynamic programming, is proposed for distributed optimal leader-following consensus of heterogeneous nonlinear multi-agent systems under directed communication graph. This scheme combines game theory, adaptive dynamic programming together with generalized fuzzy hyperbolic model (GFHM). Firstly, based on precompensation technique, an appropriate model transformation is proposed to convert the error system into augmented error system, and an exquisite performance index function is defined for this system. Secondly, on the basis of Hamilton–Jacobi–Bellman (HJB) equation, the optimal consensus control is designed and a novel policy iteration (PI) algorithm is put forward to learn the solutions of the HJB equation online. Here, the proposed PI algorithm is implemented on account of GFHMs. Compared with dual-network model including critic network and action network, the proposed scheme only requires critic network. Thirdly, the augmented consensus error of each agent and the weight estimation error of each GFHM are proved to be uniformly ultimately bounded, and the stability of our method has been verified. Finally, some numerical examples and application examples are conducted to demonstrate the effectiveness of the theoretical results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. VanDyke M, Hall C (2006) Decentralized coordinated attitude control within a formation of spacecraft. J Guidance Control Dyn 29(5):1101–1109

    Google Scholar 

  2. Wu J, Han D, Liu K et al (2015) Nonlinear suboptimal synchronized control for relative position and relative attitude tracking of spacecraft formation flying. J Frankl Inst 352(4):1495–1520

    MathSciNet  MATH  Google Scholar 

  3. Shahbazi B, Malekzadeh M, Koofigar H (2017) Robust constrained attitude control of spacecraft formation flying in the presence of disturbances. IEEE Trans Aerosp Electron Syst 53(5):2534–2543

    Google Scholar 

  4. Jiang Y, Wang H, Wang S (2019) Distributed H\(\infty\) consensus control for nonlinear multi-agent systems under switching topologies via relative output feedback. Neural Comput Appl 31(1):1–9

    MathSciNet  Google Scholar 

  5. Cai Y, Zhang H, Zhang K et al (2019) Distributed leader-following consensus of heterogeneous second-order time-varying nonlinear multi-agent systems under directed switching topology. Neurocomputing 325:31–47

    Google Scholar 

  6. Miao G, Ma Q, Liu Q (2016) Consensus problems for multi-agent systems with nonlinear algorithms. Neural Comput Appl 27(5):1327–1336

    Google Scholar 

  7. Sun Q, Han R, Zhang H et al (2015) A multiagent-based consensus algorithm for distributed coordinated control of distributed generators in the energy internet. IEEE Trans Smart Grid 6(6):3006–3019

    Google Scholar 

  8. Abdulghafor R, Abdullah S, Turaev S et al (2018) Linear and nonlinear stochastic distribution for consensus problem in multi-agent systems. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3615-x

    Article  Google Scholar 

  9. Wang B, Chen W, Wang J et al (2018) Cooperative tracking control of multiagent systems: a heterogeneous coupling network and intermittent communication framework. IEEE Trans Cybern 99:1–13

    Google Scholar 

  10. Cui B, Zhao C, Ma T et al (2016) Leader-following consensus of nonlinear multi-agent systems with switching topologies and unreliable communications. Neural Comput Appl 27(4):909–915

    Google Scholar 

  11. Deng C, Yang G (2019) Distributed adaptive fault-tolerant control approach to cooperative output regulation for linear multi-agent systems. Automatica 103:62–68

    MathSciNet  MATH  Google Scholar 

  12. Chen C, Ren C, Du T (2016) Fuzzy observed-based adaptive consensus tracking control for second-order multiagent systems with heterogeneous nonlinear dynamics. IEEE Trans Fuzzy Syst 24(4):906–915

    Google Scholar 

  13. Zhang J, Wang Z, Zhang H et al (2018) Data-based optimal control of multiagent systems: a reinforcement learning design approach. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2018.2868715

  14. Qiu Z, Liu S, Xie L (2016) Distributed constrained optimal consensus of multi-agent systems. Automatica 68:209–215

    MathSciNet  MATH  Google Scholar 

  15. Morris P (2012) Introduction to game theory. Springer Science and Business Media, New York

    MATH  Google Scholar 

  16. Vamvoudakis K, Lewis F, Hudas G (2012) Multi-agent differential graphical games: online adaptive learning solution for synchronization with optimality. Automatica 48(8):1598–1611

    MathSciNet  MATH  Google Scholar 

  17. Wei Q, Liu D, Lewis F (2015) Optimal distributed synchronization control for continuous-time heterogeneous multi-agent differential graphical games. Inf Sci 317:96–113

    MATH  Google Scholar 

  18. Sun J, Liu C (2018) Distributed zero-sum differential game for multi-agent systems in strict-feedback form with input saturation and output constraint. Neural Netw 106:8–19

    MATH  Google Scholar 

  19. Al-Tamimi A, Lewis F, Abu-Khalaf M (2008) Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof. IEEE Trans Syst Man Cybern B (Cybern) 38(4):943–949

    Google Scholar 

  20. Bertsekas D (2017) Value and policy iterations in optimal control and adaptive dynamic programming. IEEE Trans Neural Netw Learn Syst 28(3):500–509

    MathSciNet  Google Scholar 

  21. Wei Q, Liu D, Lin H (2016) Value iteration adaptive dynamic programming for optimal control of discrete-time nonlinear systems. IEEE Trans Cybern 46(3):840–853

    Google Scholar 

  22. Gao W, Jiang Y, Davari M (2019) Data-driven cooperative output regulation of multi-agent systems via robust adaptive dynamic programming. IEEE Trans Circuits Syst II Express Briefs 66(3):447–451

    Google Scholar 

  23. Zhang H, Liang H, Wang Z et al (2017) Optimal output regulation for heterogeneous multiagent systems via adaptive dynamic programming. IEEE Trans Neural Netw Learn Syst 28(1):18–29

    Google Scholar 

  24. Liu L, Wang Z, Zhang H (2018) Data-based adaptive fault estimation and fault tolerant control for MIMO model-free systems using generalized fuzzy hyperbolic model. IEEE Trans Fuzzy Syst 26(6):3191–3205

    Google Scholar 

  25. Zhang H, Wang Z, Li M et al (2004) Generalized fuzzy hyperbolic model: a universal approximator. ACTA Autom Sin 30(3):416–422

    MathSciNet  Google Scholar 

  26. Yang Z, Zhang H (2018) A fuzzy adaptive tracking control for a class of uncertain strick-feedback nonlinear systems with dead-zone input. Neurocomputing 272:130–135

    Google Scholar 

  27. Ferrari-Trecate G, Galbusera L, Marciandi M et al (2009) Model predictive control schemes for consensus in multi-agent systems with single-and double-integrator dynamics. IEEE Trans Autom Control 54(11):2560–2572

    MathSciNet  MATH  Google Scholar 

  28. Song R, Lewis F, Wei Q et al (2016) Off-policy actor-critic structure for optimal control of unknown systems with disturbances. IEEE Trans Cybern 46(5):1041–1050

    Google Scholar 

  29. Wang W, Chen X (2018) Model-free optimal containment control of multi-agent systems based on actor-critic framework. Neurocomputing 314:242–250

    Google Scholar 

  30. Zhang H, Feng T, Yang G et al (2015) Distributed cooperative optimal control for multiagent systems on directed graphs: an inverse optimal approach. IEEE Trans Cybern 45(7):1315–1326

    Google Scholar 

  31. Zhao W, Li R, Zhang H (2017) Leader-follower optimal coordination tracking control for multi-agent systems with unknown internal states. Neurocomputing 249:171–181

    Google Scholar 

  32. Yasini S, Sitani M, Kirampor A (2016) Reinforcement learning and neural networks for multi-agent nonzero-sum games of nonlinear constrained-input systems. Int J Mach Learn Cybern 7(6):967–980

    Google Scholar 

  33. Chen K, Wang J, Zhang Y et al (2015) Consensus of second-order nonlinear multi-agent systems under state-controlled switching topology. Nonlinear Dyn 81(4):1871–1878

    MathSciNet  MATH  Google Scholar 

  34. Wang A, Liao X, Dong T (2018) Event-driven optimal control for uncertain nonlinear systems with external disturbance via adaptive dynamic programming. Neurocomputing 281:188–195

    Google Scholar 

  35. Zhang H, Yue D, Zhao W et al (2018) Distributed optimal consensus control for multiagent systems with input delay. IEEE Trans Cybern 48(6):1747–1759

    Google Scholar 

  36. Das A, Lewis F (2010) Distributed adaptive control for synchronization of unknown nonlinear networked systems. Automatica 46(12):2014–2021

    MathSciNet  MATH  Google Scholar 

  37. Murray J, Cox C, Lendaris G et al (2002) Adaptive dynamic programming. IEEE Trans Syst Man Cybern Part C (Appl Rev) 32(2):140–153

    Google Scholar 

  38. Surhone L, Timpledon M, Marseken S et al (2010) Weierstrass theorem. Betascript Publishing, Riga

    Google Scholar 

  39. Vamvoudakis K, Lewis F (2010) Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5):878–888

    MathSciNet  MATH  Google Scholar 

  40. Wang D, Liu D, Li H (2014) Policy iteration algorithm for online design of robust control for a class of continuous-time nonlinear systems. IEEE Trans Autom Sci Eng 11(2):627–632

    Google Scholar 

  41. Dong B, Zhou F, Liu K et al (2018) Decentralized robust optimal control for modular robot manipulators via critic-identifier structure-based adaptive dynamic programming. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3714-8

    Article  Google Scholar 

  42. Zhang H, Cui L, Luo Y (2013) Near-optimal control for nonzero-sum differential games of continuous-time nonlinear systems using single-network ADP. IEEE Trans Cybern 43(1):206–216

    Google Scholar 

  43. Liang Y, Zhang H, Xiao G et al (2018) Reinforcement learning-based online adaptive controller design for a class of unknown nonlinear discrete-time systems with time delays. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3537-7

    Article  Google Scholar 

  44. Yang X, Liu D, Wei Q et al (2016) Guaranteed cost neural tracking control for a class of uncertain nonlinear systems using adaptive dynamic programming. Neurocomputing 198:80–90

    Google Scholar 

  45. Khalil H (2002) Nonlinear systems, 3rd edn. Prentice-Hall, Englewood Cliffs

    MATH  Google Scholar 

  46. Mei J, Ren W, Ma G (2013) Distributed coordination for second-order multi-agent systems with nonlinear dynamics using only relative position measurements[J]. Automatica 49(5):1419–1427

    MathSciNet  MATH  Google Scholar 

  47. Ren W (2008) Synchronization of coupled harmonic oscillators with local interaction. Automatica 44(12):3195–3200

    MathSciNet  MATH  Google Scholar 

  48. Du H, He Y, Cheng Y (2014) Finite-time synchronization of a class of second-order nonlinear multi-agent systems using output feedback control. IEEE Trans Circuits Syst I Reg Pap 61(6):1778–1788

    Google Scholar 

  49. Modares H, Lewis F, Kang W et al (2018) Optimal synchronization of heterogeneous nonlinear systems with unknown dynamics. IEEE Trans Autom Control 63(1):117–131

    MathSciNet  MATH  Google Scholar 

  50. He Q, Wang X, Lei Z et al (2019) TIFIM: A Two-stage iterative framework for influence maximization in social networks. Appl Math Comput 354(1):338–352

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (61433004, 61627809, 61621004), and IAPI Fundamental Research Funds 2013ZCX14.

Author information

Authors and Affiliations

  1. State Key Laboratory of Synthetical Automation for Process Industries, College of Information Science and Engineering, Northeastern University, Shenyang, Liaoning, China

    Yuliang Cai, Huaguang Zhang, Kun Zhang & Chong Liu

Authors
  1. Yuliang Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huaguang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huaguang Zhang.

Ethics declarations

Conflict of interest

The authors declare that there are no financial and personal relationships with other people or organizations that can inappropriately influence our work. And there are no potential conflicts of interest with respect to this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cai, Y., Zhang, H., Zhang, K. et al. Fuzzy adaptive dynamic programming-based optimal leader-following consensus for heterogeneous nonlinear multi-agent systems. Neural Comput & Applic 32, 8763–8781 (2020). https://doi.org/10.1007/s00521-019-04263-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04263-0

Keywords

Search

Navigation