Fault tolerant cooperative control for affine multi-agent systems: An optimal control approach

https://doi.org/10.1016/j.jfranklin.2018.09.038Get rights and content

Abstract

The goal of this paper is to propose an optimal fault tolerant control (FTC) approach for multi-agent systems (MASs). It is assumed that the agents have identical affine dynamics. The underlying communication topology is assumed to be a directed graph. The concepts of both inverse optimality and partial stability are further employed for designing the control law fully developed in the paper. Firstly, the optimal FTC problem for linear MASs is formulated and then it is extended to MASs with affine nonlinear dynamics. To solve the Hamilton-Jacobi-Bellman (HJB) equation, an Off-policy Reinforcement Learning is used to learn the optimal control law for each agent. Finally, a couple of numerical examples are provided to demonstrate the effectiveness of the proposed scheme.

Introduction

Recently, the optimal control problem for MASs has received a compelling attention in various engineering communities. As the cornerstone, most studies have been conducted with linear dynamics [1], [2], [3], [4], [5]. However, in practice, a broad class of systems and vehicles cannot be expressed exactly by linear model.Therefore, the study of nonlinear systems is essential.

On the other hand, faults in automated processes can highly effect on the performance of control systems. Therefore, the FTC is highly required for modern complex control systems. In [6], [7], [8], for particular systems, some important aspects of FTC were presented. For instance, in [6], for the LQR design, the Riccati Algebraic equation in the Electric Vehicles was solved for both fault-free and faulty conditions. However, this approach suffers from the controller switches. To put it in another way, in this approach the time at which the fault is detected plays an important role in success of the FTC. Hence, if the fault is not detected at the suitable time, this method will fail to overcome the fault. Linearizing the flying system around its equilibrium point, the authors in [7] solved the actuator FTC for that. In [8], using feedback linearization and model predictive control techniques, the control strategy is proposed in the fault-free mode, then in fault situation two different plans have been used. In the first plan, it removes faulty robots from the mission. Afterwards, in the second plan, in the case in which the faulty robot is able to be resistant, it reconfigures the controller for fault correction. In [9], to solve loss of actuator effectiveness, two performance indices were considered, one with fault (actuator failure) and the other without it. However, in this paper, one performance index for solving the actuator loss of efficiency problem in MASs is introduced. The proposed approach is more straightforward solution than the approach formulated in [9]. In [10] a robust FTC was utilized for special case of second-order nonlinear SISO systems. Moreover, [11] considered an actuator fault for special class of SISO affine nonlinear systems, and relying on the performance index, proposed an adaptive FTC framework. It is worth mentioning that the proposed FTC can be employed to general multi-input affine nonlinear systems. Emphasizing the actuator fault importance, some other researchers [12], [13], [14] investigated the FTC for some class of affine nonlinear systems.

An active fault tolerant approach required real time estimation of loss efficiency. In [15], proposed a fault estimation strategy based on iterative mean estimation technique for the linear systems with actuator faults and disturbances which provide the most up-to-date information about the true system status induced changes. None of the references mentioned above considered the FTC for MASs. Based on the authors knowledge FTC for MASs is still a challenging issue.

The FTC problem is of crucial importance specially in MASs. It is due to the fact that if an agent encounters a fault, it may result in a whole mission failure. In this field some studies have been done on the design of FTC scheme for MASs in the recent years [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32]. In [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], researchers considered an actuator fault for linear MASs. Furthermore, in some of them an uncertainty term was added to the linear system in order to make the model more practical.

The approaches in [18], [19], [20], [21], [22], [23], [24], [25] only suitable to undirected graphs. The directed graph, compared with undirected graph, is more general. The reason is that undirected graphs can be treated as a special case of directed graphs. However, the proposed approach in this study can be applied for MAS with directed graph. Papers [26], [27], [28], [29], [30], [31], [32] made a further progress and developed actuator FTC problem for particular non-linear MASs, that is a linear system with an additive non-linear term. Despite the papers in which merely SISO systems [26], [27], [28], [29], [30], [31], or undirected graphs were considered, in this paper, we put forward a general form of affine nonlinear MASs with multi-input actuator faults for directed communication topologies. The fault tolerant performance index considered in this paper for leader-less and leader-follower MASs is applicable for either directed or undirected graph. In [33] as well as many other papers on Fault-Tolerant Consensus of MASs only investigated the linear MASs under undirected topology, which can be regarded as a special case. However, in our work, we have considered nonlinear MASs which is more practical and also have relaxed the condition on the network topologies, i.e., directed topologies.

The fault considered in this paper is assumed to of loss of effectiveness type. Based on partial stability concepts and classical optimal control, an optimal approach for FTC of MASs is proposed. It should be noted that a directed graph communication topology is considered among the agents. Moreover, we design the distributed control for both general linear and affine nonlinear systems. In order to find the distributed optimal FTC of affine nonlinear MASs, Reinforcement learning (RL) is employed to solve the HJB equation for each agent.

In this research, it is proposed a performance index which attempts to reach a consensus in both normal condition systems and a faulty one. we focus on developing an optimal approach in FTC. On the other words, the proposed approach which is applicable to both linear and non-linear systems. Furthermore, this approach can be applied for both leaderless and leader-follower control frameworks. The proposed method is such that in the occurrence of a time-variant fault, the controller would be designed based on the worst fault condition. The extremes of the worst tolerable fault for the controller depend on the actuators abilities. The main contributions are as follows:

  • 1.

    The proposed optimal control law for homogeneous affine nonlinear MASs with directed communication topologies is robust against time varying actuator fault.

  • 2.

    It is developed a model-free RL algorithm to solve Bellman equation of each agent.

In view of the above discussion, the rest of this paper is organized in 6 sections. In Section 2, Background and Preliminaries are presented. In Section 3, the optimal FTC is proposed. In Section 4, a Off-line RL Algorithm to Solve the HJB equation is extended. In Section 5, two illustration examples are given to demonstrate the validity of the theoretical results. Finally, Section 6 draws the conclusion and suggestions for future study.

Section snippets

Notation

In this paper, In denotes the identity matrix of order N and let λmax{} and λmin{} represents the largest and smallest eigenvalue of a matrix respectively. ⊗ denote the Kronecker product. Vx is the gradient of a V(x) with respect to x. diag shows that is diagonal matrix. blockdiag denotes a square diagonal matrix in which the diagonal elements are square matrices. The Frobenius norm shown as ∥.∥. Identity matrix of order m is denoted by Im.

Graph properties

Let G=(V,E,A) be a weighted digraph of order N, with

Optimal FTC

In the following, we define a performance index which results in fault tolerant consensuses for linear and nonlinear MASs. moreover, the Lyapunov function is the same as performance index. An optimal problem with classical results on optimality state can be solved by considering the Hamiltonian [34], [35].

Off-line RL algorithm to solve the HJB equation

To solve HJB equation, two approaches can be taken: model-based approach such as [36], [37], [38] which exploits the bellman equation and model-free approach by [39], [40], [41] which an approximate solve is presented relying on performance index. Model-free methods are based on RL. In this paper, the second approach to deal with the HJB equation is considered. Defining performance index as a function of local neighbour error, the offline RL algorithm [40] is developed based on the information

Simulation results

In this section, two examples are presented to illustrate the effectiveness of the proposed approach.

Example 5.1

In this example, a ship autopilot dynamic is provided [11]. Nonlinear model of the ship described byx˙1=x2x˙2=4(x2+x23+x25)+4x3x˙3=8x3+4u

The communication topology is shown in Fig. 1 and the agents are initialized from:x1,0(t)=[3.20.51.0]T,x2,0(t)=[2.21.01.0]T,x3,0(t)=[1.51.62.0]T,x4,0(t)=[0.50.52.0]TThen for the purpose of simulation, we assume that an actuator fault occurs in the

Conclusion

In this paper, the FTC problem for MAS with general dynamics and directed underlying network topology has been considered. By introducing a performance index, we proposed a control scheme for a group of nonlinear affine systems with actuator failure guaranteeing robustness against time-varying fault has been designed. In order to present the solution for Bellman equation which were used to solve synchronization consensus, we exploited RL algorithm.

Extending the proposed design technique to the

References (41)

  • K.H. Movric et al.

    Cooperative optimal control for multi-agent systems on directed graph topologies

    IEEE Trans. Automat. Control

    (2014)
  • ZhangH. et al.

    Distributed cooperative optimal control for multiagent systems on directed graphs: an inverse optimal approach

    IEEE Trans. Cybern.

    (2015)
  • M.E. Dehshalie et al.

    Finite-time distributed global optimal control for linear time-varying multi-agent systems: a dynamic output-feedback perspective

    IET Control Theory Appl.

    (2018)
  • F.L. Lewis et al.

    Cooperative Control of Multi-agent Systems: Optimal and Adaptive Design Approaches

    (2013)
  • YangH. et al.

    Optimal fault-tolerant path-tracking control for 4ws4wd electric vehicles

    IEEE Trans. Intell. Transp. Syst.

    (2010)
  • YuX. et al.

    Fault-tolerant flight control design with finite-time adaptation under actuator stuck failures

    IEEE Trans. Control Syst. Technol.

    (2017)
  • M.A. Kamel et al.

    Fault-tolerant cooperative control design of multiple wheeled mobile robots

    IEEE Trans. Control Syst. Technol.

    (2018)
  • XieC.-H. et al.

    Approximate guaranteed cost fault-tolerant control of unknown nonlinear systems with time-varying actuator faults

    Nonlinear Dyn.

    (2016)
  • VanM. et al.

    Robust fault-tolerant control for a class of second-order nonlinear systems using an adaptive third-order sliding mode control

    IEEE Trans. Syst. Man Cybern. Syst.

    (2017)
  • ZhangJ.-X. et al.

    Robust adaptive fault-tolerant control for a class of unknown nonlinear systems

    IEEE Trans. Ind. Electr.

    (2017)
  • Cited by (46)

    • Adaptive consensus control of second-order nonlinear multi-agent systems with event-dependent intermittent communications

      2023, Journal of the Franklin Institute
      Citation Excerpt :

      Coordinated control of multi-agent systems (MASs) has received wide concerns in last decade because their wide applications in unmanned systems, spacecraft formation, under water vehicles and so on [1–6].

    • Resilient control of double-integrator stochastic multi-agent systems under denial-of-service attacks

      2022, Journal of the Franklin Institute
      Citation Excerpt :

      The security of multi-agent systems can be divided into physical security and network security [9]. The works [10–13] respectively on the actuator failure, sensor failure, and actuator and sensor failure are studied. The common network attacks are DoS attacks [14–16], deception attacks [17,18] and replay attacks [19,20].

    • Distributed fault-tolerant tracking control for heterogeneous nonlinear multi-agent systems under sampled intermittent communications

      2021, Journal of the Franklin Institute
      Citation Excerpt :

      For the distributed FTC control problem of MASs, it is more difficult than the FTC of a single agent because they communicate with each other [31–33]. In [34], an optimal control law for affine nonlinear MASs under directed topologies and actuator faults is developed. Based on the control allocation and optimal method, a distributed FTC strategy is proposed for linear MASs [35].

    View all citing articles on Scopus
    View full text